Reconstruction-based anomaly detection
Methods and systems for detecting and correcting anomalies include predicting normal behavior of a monitored system based on training data that includes only sensor data collected during normal behavior of the monitored system. The predicted normal behavior is compared to recent sensor data to determine that the monitored system is behaving abnormally. A corrective action is performed responsive to the abnormal behavior to correct the abnormal behavior.
1. A method for detecting and correcting anomalies, comprising:
predicting normal behavior of a monitored system based on training data that includes only sensor data collected during normal behavior of the monitored system, by generating time-series data that represents the predicted normal behavior of the monitored system using an autoencoder model trained with the training data;
receiving time-series data that represents recent sensor measurements;
comparing the time-series data representing the predicted normal behavior to the time-series data representing the recent sensor data to determine that the monitored system is behaving abnormally; and
performing a corrective action responsive to the abnormal behavior to correct the abnormal behavior.
2. The method of claim 1 , wherein the autoencoder model is a neural network autoencoder and comprises an encoder part and a decoder part.
3. The method of claim 2 , wherein the encoder part has an input and an output, where a dimensionality of the encoder's output is lower than a dimensionality of the encoder's input.
4. The method of claim 1 , wherein predicting normal behavior comprises masking part of the training data and reconstructing the masked part.
5. The method of claim 4 , wherein masking part of the training data comprises removing values from time series that make up the training data.
6. The method of claim 1 , wherein comparing comprises comparing the recent sensor data to the predicted normal behavior using a comparison function and determining that a comparison result value is above a threshold.
7. The method of claim 6 , wherein comparing comprises determining a mean of squared vector differences between the predicted normal behavior and the recent sensor data.
8. The method of claim 6 , wherein the comparison function is based on domain-specific heuristics.
9. The method of claim 1 , wherein the corrective action is selected from the group consisting of changing a security setting for an application or hardware component of the monitored system, changing an operational parameter of an application or hardware component of the monitored system, halting or restarting an application of the monitored system, halting or rebooting a hardware component of the monitored system, changing an environmental condition of the monitored system, and changing status of a network interface of the monitored system.
10. A system for detecting and correcting anomalies, comprising:
a machine learning model that includes an autoencoder model trained with training data that includes only sensor data collected during normal behavior of the monitored system, configured to predict normal behavior of a monitored system with output time-series data that represents the predicted normal behavior of the monitored system;
an anomaly module comprising a processor configured to compare the predicted normal behavior to recent sensor data to determine that the monitored system is behaving abnormally; and
a control module configured to perform a corrective action responsive to the abnormal behavior to correct the abnormal behavior.
11. The system of claim 10 , wherein the autoencoder model is a neural network autoencoder and comprises an encoder part and a decoder part.
12. The system of claim 11 , wherein the encoder part has an input and an output, where a dimensionality of the encoder's output is lower than a dimensionality of the encoder's input.
13. The system of claim 10 , wherein the machine learning model is further configured to mask part of the training data and to reconstruct the masked part.
14. The system of claim 13 , wherein the machine learning model is further configured to remove values from time series that make up the training data to mask part of the training data.
15. The system of claim 10 , wherein the anomaly module is further configured to compare the recent sensor data to the predicted normal behavior using a comparison function and to determine that a comparison result value is above a threshold.
16. The system of claim 15 , wherein the anomaly module is further configured to determine a mean of squared vector differences between the predicted normal behavior and the recent sensor data.
17. The system of claim 15 , wherein the comparison function is based on domain-specific heuristics.
18. The system of claim 10 , wherein the corrective action is selected from the group consisting of changing a security setting for an application or hardware component of the monitored system, changing an operational parameter of an application or hardware component of the monitored system, halting or restarting an application of the monitored system, halting or rebooting a hardware component of the monitored system, changing an environmental condition of the monitored system, and changing status of a network interface of the monitored system.