Intelligently determining when to perform enhanced logging
In general, in one aspect, the invention relates to a method for enabling enhanced logging. The method includes obtaining a log associated with a job; determining, using the log, that enhanced logging is to be enabled prior to initiating the job; enabling, in response to the determination, enhanced logging on at least one node, and initiating servicing of the job, after the enabling, on the at least one node.
1. A method for enabling enhanced logging, comprising:
obtaining a log associated with a job;
determining, using the log, that enhanced logging is to be enabled prior to initiating the job, wherein determining, using the log, that enhanced logging is to be enabled prior to initiating the job comprises:
detecting an error in the log, and
matching the error to an error entry of a plurality of error entries in an accept storage, wherein the plurality of error entries correspond to errors that predict a likely failure of the job;
enabling, in response to the determination, enhanced logging on at least one node; and
initiating servicing of the job, after the enabling, on the at least one node,
wherein the log includes a minimum set of data and enabling enhanced logging comprises of logging a larger set of data than the minimum set of data.
2. The method of claim 1 , wherein the job is associated with a plurality of microservices, wherein the log is one of a plurality of logs associated with at least one of the plurality of microservices, and wherein the initiating the servicing of the job comprises initiating the servicing of at least a portion of the job on the at least one of the plurality of microservices.
3. The method of claim 1 , wherein the log comprises historical microservice data, wherein the historical microservice data is associated with a microservice that serviced at least a portion of a second job, wherein the second job is a same type of job as the job.
4. The method of claim 1 , wherein the log specifies an error and wherein the error matches an error entry in an accept storage.
5. The method of claim 4 , wherein the error entry in the accept storage specifies the error, wherein the error is associated with a second job and wherein the second job failed.
6. The method of claim 5 , wherein the second job is a same type of job as the job.
7. The method of claim 1 , wherein the log comprises live microservice data, wherein the live microservice data is associated with a microservice that will service at least a portion of the job.
8. The method of claim 7 , wherein the live microservice data is obtained in real-time or near real-time from the microservice.
9. The method of claim 1 , wherein the job is a backup job.
10. A method for enabling enhanced logging, comprising:
obtaining a log associated with a job;
making a first determination, using the log, not to enable enhanced logging prior to initiating the job;
initiating, based on the first determination, servicing of the job, after the enabling, on at least one node, wherein making the first determination, using the log, not to enable enhanced logging prior to initiating the job comprises:
detecting an error in the log, and
matching the error to an error entry in a reject storage,
wherein the error is associated with a second job,
wherein prior to the second job being successfully completed, the error entry was located in the accept storage, and
wherein after the job was successfully completed, the error entry was moved to the reject storage;
after the initiating, making a second determination, using a second log, that enhanced logging is to be enabled;
in response to the second determination:
stopping the servicing of the job;
enabling, after the stopping, enhanced logging on the at least one node; and
re-initiating servicing of the job, after the enabling, on the at least one node,
wherein the log includes a minimum set of data and enabling enhanced logging comprises of logging a larger set of data than the minimum set of data.
11. The method of claim 10 , wherein the second log is obtained from at least one microservice while it is servicing at least a portion of the job prior to the second determination.
12. The method of claim 10 , wherein making the second determination, using the second log, that enhanced logging is to be enabled comprises:
detecting an error in the second log; and
matching the error to an error entry in an accept storage, wherein the error entry in the accept storage specifies the error, wherein the error is associated with a second job and wherein the second job failed.
13. The method of claim 12 , wherein the second log comprises live microservice data, wherein the live microservice data is associated with a microservice executing on the at least one node.
14. The method of claim 13 , wherein the live microservice data is obtained in real-time or near real-time from the microservice.
15. The method of claim 12 , wherein the second job is a same type of job as the job.
16. The method of claim 10 , wherein the job is associated with a plurality of microservices, wherein the log is one of a plurality of logs associated with at least one of the plurality of microservices.
17. The method of claim 10 , wherein the job is a backup job.
18. The method of claim 10 , wherein obtaining the log associated with the job comprises using a microservice correlation map.