Dynamic window-size selection for anomaly detection
Detecting system log anomalies by receiving multivariate time-series system log data an multivariate metric data, inferring system metrics from the system log data and metric data, receiving a metric causal graph including causal relationships between system metrics, determining a univariate variation score for the system metrics, determining a causal variation score for the multivariate time series system metric data, according to the causal graph, determining an overall activity score according to the univariate variation score, and causal variation score, and altering a review window duration according to the activity score.
1. A computer implemented method for system log anomaly detection by:
receiving, by one or more computer processors, multivariate time-series system log data and metric data;
receiving, by the one or more computer processors, a metric data causal graph including causal relationships between system metrics;
determining, by the one or more computer processors, a univariate variation score for the system metrics;
determining, by the one or more computer processors, a causal variation score for the multivariate time series system metric data according to the causal graph;
determining, by the one or more computer processors, an activity score according to the univariate variation score, and causal variation score;
altering, by the one or more computer processors, a review window duration according to the activity score; and
processing, by the one or more computer processors, windows of multivariate time-series system log data and metric data to detect anomalies.
2. The computer implemented method according to claim 1 , further comprising:
determining, by the one or more computer processors, a metrics anomaly score according to a metric-based anomaly detector; and
determining, by the one or more computer processors, the activity score according to the metrics anomaly score, univariate variation score, and causal variation score.
3. The computer implemented method according to claim 1 , wherein the univariate variation score comprises metric velocity scores and metric acceleration scores.
4. The computer implemented method according to claim 1 , wherein the review window duration is inversely proportional to the activity score.
5. The computer implemented method according to claim 1 , further comprising determining the univariate variation score for the system metrics according to a metric velocity score, wherein the metric velocity score comprises a mode of the metric velocity.
6. The computer implemented method according to claim 1 , further comprising determining the univariate variation score for the system metrics according to a metric acceleration score, wherein the metric acceleration score comprises a mode of the metric acceleration.
7. The computer implemented method according to claim 1 , further comprising detecting, by the one or more computer processors, system log data anomalies within the review window.
8. A computer program product for system log anomaly detection, the computer program product comprising one or more computer readable storage media and collectively stored program instructions on the one or more computer readable storage media, the stored program instructions comprising:
program instructions to receiving multivariate time-series system log data and metric data;
program instructions to receive a metric data causal graph including causal relationships between system metrics;
program instructions to determine a univariate variation score for the system metrics;
program instructions to determine a causal variation score for the multivariate time series system metric data according to the causal graph;
program instructions to determine an activity score according to the univariate variation score, and causal variation score;
program instructions to alter a review window duration according to the activity score; and
program instructions to process windows of multivariate time-series system log data and metric data to detect anomalies.
9. The computer program product according to claim 8 , the stored program instructions further comprising:
program instructions to determine a metrics anomaly score according to a metric-based anomaly detector; and
program instructions to determine the activity score according to the metrics anomaly score, univariate variation score, and causal variation score.
10. The computer program product according to claim 8 , wherein the univariate variation score comprises metric velocity scores and metric acceleration scores.
11. The computer program product according to claim 8 , wherein the review window duration is inversely proportional to the activity score.
12. The computer program product according to claim 8 , further comprising determining the univariate variation score for the system metrics according to a metric velocity score, wherein the metric velocity score comprises a mode of the metric velocity.
13. The computer program product according to claim 8 , further comprising determining the univariate variation score for the system metrics according to a metric acceleration score, wherein the metric acceleration score comprises a mode of the metric acceleration.
14. The computer program product according to claim 8 , the stored program instructions further comprising program instructions to detect system log data anomalies within the review window.
15. A computer system for detecting system log anomalies, the computer system comprising:
one or more computer processors;
one or more computer readable storage devices; and
stored program instructions on the one or more computer readable storage devices for execution by the one or more computer processors, the stored program instructions comprising:
program instructions to receiving multivariate time-series system log data and metric data;
program instructions to infer system metrics from the system log and metric data;
program instructions to receive a metric causal graph including causal relationships between system metrics;
program instructions to determine a univariate variation score for the system metrics;
program instructions to determine a causal variation score for the multivariate time series system
metric data according to the causal graph;
program instructions to determine an activity score according to the univariate variation score, and causal variation score;
program instructions to alter a review window duration according to the activity score; and
program instructions to process windows of multivariate time-series system log data and metric data to detect anomalies.
16. The computer system according to claim 15 , the stored program instructions further comprising:
program instructions to determine a metrics anomaly score according to a metric-based anomaly detector; and
program instructions to determine the activity score according to the metrics anomaly score, univariate variation score, and causal variation score.
17. The computer system according to claim 15 , wherein the univariate variation score comprises metric velocity scores and metric acceleration scores.
18. The computer system according to claim 15 , wherein the review window duration is inversely proportional to the activity score.
19. The computer system according to claim 15 , further comprising determining the univariate variation score for the system metrics according to a metric velocity score, wherein the metric velocity score comprises a mode of the metric velocity.
20. The computer system according to claim 15 , further comprising determining the univariate variation score for the system metrics according to a metric acceleration score, wherein the metric acceleration score comprises a mode of the metric acceleration.