Scalable predictive early warning system for data backup event log
View Patent ↗Techniques to detect backup-related anomalies are disclosed. In various embodiments, a processor is used to generate based at least in part on backup log data associated with a training period a predictive model. The predictive model is to detect, using the processor, anomalies in corresponding backup log data associated with a detection period.
1. A method of detecting backup related anomalies, comprising:
generating, using a processor, based at least in part on backup log data associated with a training period a predictive model;
using, by the processor, the predictive model to detect, using the processor, anomalies in corresponding backup log data associated with a detection period, wherein the anomalies at least includes data being erroneously deleted;
computing a score for a detected anomaly; and
performing one or more responsive actions based at least in part on a comparison between the computed score and a detection threshold.
2. The method of claim 1 , further comprising:
filtering the backup log data associated with the training period into one or more sets of backup log data based on one or more attributes; and
extracting from a set of the one or more sets of backup log data a prescribed set of features.
3. The method of claim 2 , wherein said one or more attributes include one or more of the following: backup type; backup schedule; backup size; number of objects backed up; source system type; and other source system attribute.
4. The method of claim 2 , wherein the prescribed set of features include one or more of the following: backup size; number of objects backed up; and amount of change in backup data size.
5. The method of claim 1 , further comprising receiving via a network communication interface said backup log data associated with the training period.
6. The method of claim 4 , further comprising receiving via a network communication interface said backup log data associated with one or more backup clients during the detection period.
7. The method of claim 1 , wherein a responsive action includes the processor using backup data stored at a prior time to restore the erroneously deleted data.
8. The method of claim 1 , wherein the predictive model is associated with one or more of the following model types Gaussian hypothesis testing; KS test; and Kernel Density Estimation.
9. The method of claim 1 , wherein the predictive model is configured to be used to predict for a given set of extracted features associated with a backup performed during the detection period a corresponding statistical probability of occurrence of said given set of features.
10. The method of claim 1 , further comprising ranking detected anomalies based at least in part on their respective scores.
11. The method of claim 1 , wherein the one or more responsive actions are performed based at least in part on a determination that the computed score exceeds the detection threshold.
12. A system to detect backup related anomalies, comprising:
a communication interface; and
a processor coupled to the communication interface and configured to:
generate based at least in part on backup log data associated with a training period a predictive model; and
use the predictive model to detect, using the processor, anomalies in corresponding backup log data associated with a detection period, wherein the anomalies at least includes data being erroneously deleted;
compute a score for a detected anomaly; and
perform one or more responsive actions based at least in part on a comparison between the computed score and a detection threshold.
13. The system of claim 12 , wherein the processor is further configured to:
filter the backup log data associated with the training period into one or more sets of backup log data based on one or more attributes; and
extract from a set of the one or more sets of backup log data a prescribed set of features.
14. The system of claim 13 , wherein the prescribed set of features include one or more of the following: backup size; number of objects backed up; and amount of change in backup data size.
15. The system of claim 13 , wherein said one or more attributes include one or more of the following: backup type; backup schedule; backup size; number of objects backed up; source system type; and other source system attribute.
16. The system of claim 12 , wherein a responsive action includes the processor using backup data stored at a prior time to restore the erroneously deleted data.
17. The system of claim 12 , wherein the processor is further configured to receive via the communication interface said backup log data associated with the training period.
18. The system of claim 12 , wherein the processor is further configured to receive via the communication interface said backup log data associated with the one or more backup clients during the detection period.
19. A computer program product to detect backup related anomalies, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
generating, using a processor, based at least in part on backup log data associated with a training period a predictive model;
using, by the processor, the predictive model to detect, using the processor, anomalies in corresponding backup log data associated with a detection period, wherein the anomalies at least includes data being erroneously deleted;
computing a score for a detected anomaly; and
performing one or more responsive actions based at least in part on a comparison between the computed score and a detection threshold.
20. The computer program product of claim 19 , wherein a responsive action includes the processor using backup data stored at a prior time to restore the erroneously deleted data.