Streaming anomaly detection

1. A process stored in one or more data-storage devices and executed using one or more processors of a computer system to detect anomalous behavior exhibited by resources and objects of a distributed computing system, the process comprising:

receiving multiple streams of metric data generated by metric sources in the distributed computing system;

updating a performance model based on most recently received metric values of the streams of metric data, the performance being selected from a library of parametric, regression-based, and signal processing-based performance models;

performing anomaly detection to detect changes in one or more of the streams of metric data based on the updated performance model, the streams of metric data being assigned probability scores based on a probability density function; and

notifying an anomaly listener of anomalous behavior exhibited by a resource or object associated with a change detected in one or more of the streams of metric data.

2. The process of claim 1 wherein updating the performance model comprises:

for new metric values of the streams of metric data,

computing a mean of the recently received metric values, and

computing a sample standard deviation of the recently received metric values; and

for each new metric value of the streams of metric data, computing a standard-score model based on the recently received metric value, the mean, and the sample standard deviation.

3. The process of claim 1 wherein updating the performance model comprises:

computing a mean usage tuple from new metric values of the streams of metric data, each element of the mean-usage tuple corresponding to the mean usage of a resource of the distributed computing system;

forming a usage tuple from the new metric values of the resources;

computing a covariance matrix of the new metric values of the resources; and

computing a distance model that represents a distance from the usage tuple to the mean-usage tuple based on the usage tuple, the mean-usage tuple, and the covariance matrix.

4. The process of claim 1 wherein updating the performance model comprises:

for each stream of the multiple streams of metric data,

computing forecast metric values in a forecast interval; and

computing a forecast confidence intervals model for each of the forecast metric values.

5. The process of claim 1 wherein updating the performance model comprises:

for each stream of the streams of metric data,

determining if the stream of the metric data is a seasonal stream of metric data;

if the stream of metric data is a seasonal stream of metric data, computing a principal frequency of the stream of metric data based on new metric values in a current time window; and

computing an absolute difference between the principal frequency in the current time window and a principal frequency in a previous time window.

6. The process of claim 1 wherein performing anomaly detection to detect changes in one or more of the streams of metric data based on the updated performance model comprises:

determining a threshold based on the performance model; and

when the performance model violates the threshold, identifying the resource or object as exhibiting anomalous behavior.

7. A computer system to detect anomalous behavior exhibited by resources and objects of a distributed computing system, the system comprising:

one or more processors;

one or more data-storage devices; and

machine-readable instructions stored in the one or more data-storage devices that when executed using the one or more processors controls the system to perform operations comprising:

receiving multiple streams of metric data generated by metric sources in of the distributed computing system;

updating a performance model based on most recently received metric values of the streams of metric data, the performance being selected from a library of parametric, regression-based, and signal processing-based performance models;

performing anomaly detection to detect changes in one or more of the streams of metric data based on the updated performance model, the streams of metric data being assigned probability scores based on a probability density function; and

notifying an anomaly listener of anomalous behavior exhibited by a resource or object associated with a change detected in one or more of the streams of metric data.

8. The system of claim 7 wherein updating the performance model comprises:

for new metric values of the streams of metric data,

computing a mean of the recently received metric values, and

computing a sample standard deviation of the recently received metric values; and

for each new metric value of the streams of metric data, computing a standard-score model based on the recently received metric value, the mean, and the sample standard deviation.

9. The system of claim 7 wherein updating the performance model comprises:

computing a mean usage tuple from new metric values of the streams of metric data, each element of the mean-usage tuple corresponding to the mean usage of a resource of the distributed computing system

forming a usage tuple from the new metric values of the resources;

computing a covariance matrix of the new metric values of the resources; and

computing a distance model that represents a distance from the usage tuple to the mean-usage tuple based on the usage tuple, the mean-usage tuple, and the covariance matrix.

10. The system of claim 7 wherein updating the performance model comprises:

for each stream of the multiple streams of metric data,

computing forecast metric values in a forecast interval; and

computing a forecast confidence intervals model for each of the forecast metric values.

11. The system of claim 7 wherein updating the performance model comprises:

for each stream of the streams of metric data,

determining if the stream of the metric data is a seasonal stream of metric data;

if the stream of metric data is a seasonal stream of metric data, computing a principal frequency of the stream of metric data based on new metric values in a current time window; and

computing an absolute difference between the principal frequency in the current time window and a principal frequency in a previous time window.

12. The system of claim 7 wherein performing anomaly detection to detect changes in one or more of the streams of metric data based on the updated performance model comprises:

determining a threshold based on the performance model; and

when the performance model violates the threshold, identifying the resource or object as exhibiting anomalous behavior.

13. A non-transitory computer-readable medium encoded with machine-readable instructions that controls one or more processors of a computer system to perform the operations comprising:

receiving multiple streams of metric data generated by metric sources in a distributed computing system;

updating a performance model based on most recently received metric values of the streams of metric data, the performance being selected from a library of parametric, regression-based, and signal processing-based performance models;

performing anomaly detection to detect changes in one or more of the streams of metric data based on the updated performance model, the streams of metric data being assigned probability scores based on a probability density function; and

notifying an anomaly listener of anomalous behavior exhibited by a resource or object of the distributed computing system, the resource or object associated with a change detected in one or more of the streams of metric data.

14. The medium of claim 13 wherein updating the performance model comprises:

for new metric values of the streams of metric data,

computing a mean of the recently received metric values, and

computing a sample standard deviation of the recently received metric values; and

for each new metric value of the streams of metric data, computing a standard-score model based on the recently received metric value, the mean, and the sample standard deviation.

15. The medium of claim 13 wherein updating the performance model comprises:

computing a mean usage tuple from new metric values of the streams of metric data, each element of the mean-usage tuple corresponding to the mean usage of a resource of the distributed computing system used by the object;

forming a usage tuple from the new metric values of the resources;

computing a covariance matrix of the new metric values of the resources; and

computing a distance model that represents a distance from the usage tuple to the mean-usage tuple based on the usage tuple, the mean-usage tuple, and the covariance matrix.

16. The medium of claim 13 wherein updating the performance model comprises:

for each stream of the multiple streams of metric data,

computing forecast metric values in a forecast interval; and

computing a forecast confidence interval for each of the forecast metric values.

17. The medium of claim 13 wherein updating the performance model comprises:

for each stream of the streams of metric data,

determining if the stream of the metric data is a seasonal stream of metric data;

if the stream of metric data is a seasonal stream of metric data, computing a principal frequency of the stream of metric data based on new metric values in a current time window; and

computing an absolute difference between the principal frequency in the current time window and a principal frequency in a previous time window.

18. The medium of claim 13 wherein performing anomaly detection to detect changes in one or more of the streams of metric data based on the updated performance model comprises:

determining a threshold based on the performance model; and

when the performance model violates the threshold, identifying the resource or object as exhibiting anomalous behavior.