Anomaly detection in deduplication pruning operations
Described herein are techniques for better understanding problems arising in an illustrative information management system, such as a data storage management system, and for issuing appropriate alerts and reporting to data management professionals. The illustrative embodiments include a number of features that detect and raise awareness of anomalies in system operations, such as in deduplication pruning operations. Such anomalies can include delays in the processing of archive files to be deleted and/or delays in the generation of the list of archive files to delete. Anomalies are characterized by frequency anomalies and/or by occurrence counts. Utilization is also of interest for certain key system resources, such as deduplication databases, CPU and memory at the storage manager, etc., without limitation. Predicting low utilization periods for these and other key resources is useful for scheduling maintenance activities without interfering with ordinary deduplication pruning operations and/or other data protection jobs.
1. A networked information management system comprising:
a client computing device having one or more first hardware processors, wherein the client computing device is associated with a first job; and
one or more computing devices in communication with the client computing device, wherein the one or more computing devices are configured with computer-executable instructions that, when executed, cause the one or more computing devices to:
retrieve jobs data corresponding to the first job and the first client computing device;
perform a time-series decomposition of the jobs data;
analyze a component of the decomposed time-series to determine at least one of an acceptable range for time to perform the first job, an acceptable range for a size of secondary copy data associated with the first job, or an acceptable range for a number of job attempts until the first job is complete;
determine a possible cause for the first job running longer than the acceptable range for time to perform the first job in response to the first job at a first time running longer than the acceptable range for time to perform the first job;
determine whether any events corresponding to the first job are anomalous; and
generate an alert regarding an issue with the networked information management system in response to at least one of the first job running longer or a first event corresponding to the first job being anomalous.
2. The networked information management system of claim 1 , wherein the computer-executable instructions, when executed, further cause the one or more computing devices to perform the time-series decomposition of the jobs data to form a trend component, a seasonal component, and an error component.
3. The networked information management system of claim 2 , wherein the computer-executable instructions, when executed, further cause the one or more computing devices to analyze the error component to determine at least one of the acceptable range for time to perform the first job, the acceptable range for the size of the secondary copy data associated with the first job, or the acceptable range for the number of job attempts until the first job is complete.
4. The networked information management system of claim 1 , wherein the jobs data comprises at least one of historical data indicating a job length, secondary copy data size, or a number of job attempts for performing the first job at different time instants over a period of time.
5. The networked information management system of claim 1 , wherein the first job comprises an incremental backup job or a full backup job.
6. The networked information management system of claim 1 , wherein the possible cause comprises one of an activity being disabled, a secondary copy operation window not being enforced, a user suspending the first job, content of the secondary copy data being processed by the first job changing, or the size of the secondary copy data increasing.
7. The networked information management system of claim 1 , wherein first event comprises an event that occurs as a result of one of an action taken by the first job or a status of the first job changing.
8. The networked information management system of claim 1 , wherein the computer-executable instructions, when executed, further cause the one or more computing devices to perform a filter operation prior to generation of the alert.
9. A computer-implemented method comprising:
retrieving jobs data corresponding to a first job and a first client computing device;
performing a time-series decomposition of the jobs data;
analyzing a component of the decomposed time-series to determine at least one of an acceptable range for time to perform the first job, an acceptable range for a size of secondary copy data associated with the first job, or an acceptable range for a number of job attempts until the first job is complete;
determining a possible cause for the first job running longer than the acceptable range for time to perform the first job in response to the first job at a first time running longer than the acceptable range for time to perform the first job;
determining whether any events corresponding to the first job are anomalous; and
generating an alert regarding an issue with a networked information management system in response to at least one of the first job running longer or a first event corresponding to the first job being anomalous.
10. The computer-implemented method of claim 9 , wherein performing a time-series decomposition further comprises performing the time-series decomposition of the jobs data to form a trend component, a seasonal component, and an error component.
11. The computer-implemented method of claim 10 , wherein analyzing a component of the decomposed time-series further comprises analyzing the error component to determine at least one of the acceptable range for time to perform the first job, the acceptable range for the size of the secondary copy data associated with the first job, or the acceptable range for the number of job attempts until the first job is complete.
12. The computer-implemented method of claim 9 , wherein the jobs data comprises at least one of historical data indicating a job length, secondary copy data size, or a number of job attempts for performing the first job at different time instants over a period of time.
13. The computer-implemented method of claim 9 , wherein the first job comprises an incremental backup job or a full backup job.
14. The computer-implemented method of claim 9 , wherein the possible cause comprises one of an activity being disabled, a secondary copy operation window not being enforced, a user suspending the first job, content of the secondary copy data being processed by the first job changing, or the size of the secondary copy data increasing.
15. The computer-implemented method of claim 9 , wherein first event comprises an event that occurs as a result of one of an action taken by the first job or a status of the first job changing.
16. The computer-implemented method of claim 9 , further comprising performing a filter operation prior to generation of the alert.
17. A non-transitory computer-readable medium storing instructions, which when executed by one or more computing devices, cause the one or more computing devices to perform a method comprising:
retrieving jobs data corresponding to a first job and a first client computing device;
performing a time-series decomposition of the jobs data;
analyzing a component of the decomposed time-series to determine at least one of an acceptable range for time to perform the first job, an acceptable range for a size of secondary copy data associated with the first job, or an acceptable range for a number of job attempts until the first job is complete;
determining a possible cause for the first job running longer than the acceptable range for time to perform the first job in response to the first job at a first time running longer than the acceptable range for time to perform the first job;
determining whether any events corresponding to the first job are anomalous; and
generating an alert regarding an issue with a networked information management system in response to at least one of the first job running longer or a first event corresponding to the first job being anomalous.
18. The non-transitory computer-readable medium of claim 17 , wherein the instructions, when executed by the one or more computing devices, further cause the one or more computing devices to perform a method comprising performing the time-series decomposition of the jobs data to form a trend component, a seasonal component, and an error component.
19. The non-transitory computer-readable medium of claim 18 , wherein the instructions, when executed by the one or more computing devices, further cause the one or more computing devices to perform a method comprising analyzing the error component to determine at least one of the acceptable range for time to perform the first job, the acceptable range for the size of the secondary copy data associated with the first job, or the acceptable range for the number of job attempts until the first job is complete.
20. The non-transitory computer-readable medium of claim 17 , wherein the instructions, when executed by the one or more computing devices, further cause the one or more computing devices to perform a method comprising performing a filter operation prior to generation of the alert.