Feature vector based anomaly detection in an information technology environment

1. A method comprising:

receiving an indication of a metric corresponding to a server in a system that includes a plurality of servers, the metric including data representative of a performance or operation of a machine in an information technology environment;

computing a statistic associated with the metric;

generating a plurality of feature vectors by generating, for each of the plurality of servers, a feature vector in n-dimensional space, each of the feature vectors including a metric for a corresponding server and including the statistic associated with the metric or a spectral characteristic of the metric for the corresponding server;

computing a center of mass of the plurality of feature vectors;

determining a degree of anomaly based on a distance of the server from the center of mass; and

detecting an anomaly associated with the server based on a result of said determining.

2. The method of claim 1 , wherein the metric includes a time-stamped log entry.

3. The method of claim 1 , wherein said detecting is capable of detecting an anomaly of each of the following types: spatial environment anomaly; temporal environment anomaly, spatial behavior anomaly and temporal behavior anomaly.

4. The method of claim 1 , wherein said determining is further based on a threshold indicative of a degree of change of a configuration of the server between a first time and a second time, the method further comprising:

adjusting the threshold based on a user input received in response to a user receiving an indication of the anomaly.

5. The method of claim 1 , further comprising:

generating an alert indicative of detection of the anomaly.

6. The method of claim 1 , further comprising:

generating an alert indicative of detection of the anomaly; and

directly triggering, in response to the alert, application of a protective measure to the server, by causing transmission of a message onto the network.

7. The method of claim 1 , further comprising:

generating an alert indicative of detection of the anomaly; and

directly triggering, in response to the alert, application of a protective measure to the server, by causing transmission of a message onto the network, wherein the message causes the server to be placed into a quarantine.

8. The method of claim 1 , further comprising:

generating an alert indicative of detection of the anomaly; and

directly triggering, in response to the alert, application of a protective measure to the server, by causing transmission of a message onto the network, wherein the message comprises an HTTP POST to a throttling mechanism to quarantine a server having the anomaly.

9. The method of claim 1 , further comprising:

generating an alert indicative of detection of the anomaly; and

rating the alert based on at least one of:

an anomaly score;

a persistence of the detected anomaly; or

divergence from previous patterns.

10. The method of claim 1 , further comprising:

transforming the statistic into a particular feature vector in n-dimensional space; and

averaging values of each of the dimensions of each of the feature vectors to compute the center of mass.

11. The method of claim 1 , further comprising:

transforming the statistic into a particular feature vector in n-dimensional space; and

averaging values of each of the dimensions of each of the feature vectors to compute the center of mass; and

computing a standard deviation of the values.

12. The method of claim 1 , further comprising:

transforming the statistic into a particular feature vector in n-dimensional space;

determining a set of values corresponding to each of the dimensions of each of the feature vectors;

computing a standard deviation of the values; and

adjusting a threshold to a value of a smallest standard deviation value that has a specified degree of influence on the standard deviation of the set, the threshold for use in said determining a degree of anomaly.

13. The method of claim 1 , further comprising:

transforming the statistic into a particular feature vector in n-dimensional space;

determining a set of data values corresponding to each of the dimensions of each of the feature vectors;

dividing the data into quartiles; and

adjusting a threshold to a sum of a lower quartile value, and a product of an inter-quartile range and a gamma value based on a degree of skew of the data, the threshold for use in said determining a degree of anomaly.

14. The method of claim 1 , further comprising computing a true positive ratio and a false positive ratio for measuring an effectiveness of anomaly detection.

15. The method of claim 1 , further comprising:

determining a number of false positive anomaly detections; and

when the number exceeds a specified value, adjusting automatically a threshold for use in said determining a degree of anomaly.

16. The method of claim 1 , wherein the metric is selected from a group comprising real-time metrics, and replays of metrics collected prior to the start of a specified time period.

17. The method of claim 1 , wherein the metric is selected from a group consisting of memory usage, CPU usage, traffic and load.

18. The method of claim 1 , further comprising calculating statistics for the metric for each of the plurality of servers for each of multiple time windows, wherein the statistics are selected from a group consisting of mean, standard deviation, kurtosis and entropy.

19. The method of claim 1 , further comprising calculating spectral characteristics of the metric, wherein the spectral characteristics are selected from a group consisting of wavelet coefficients and Fourier coefficients.

20. The method of claim 1 , further comprising:

averaging values of each dimension of each of the feature vectors to compute the center of mass, wherein the distance measurement from a feature vector of a first server of the plurality of servers to the center of mass is one selected from a group consisting of Euclidean distance, L1 (Manhattan) distance, and Canberra distance.

21. The method of claim 1 , wherein the metric comprises a synchronous time series metric.

22. The method of claim 1 , wherein the metric comprises a synchronous time series metric comprising a series of time stamp and value tuples generated at regular intervals.

23. The method of claim 1 , wherein the metric comprises a synchronous time series metric comprising a series of time stamp and value tuples generated at regular intervals, and wherein the synchronous time series metric comprises a number of user requests per second, and a CPU load per second.

24. The method of claim 1 , wherein the metric comprises a synchronous time series metric comprising:

a synchronous time series of average packet size value per second;

a maximum average packet size value over a time interval; and

a minimum average packet size value over the time interval.

25. The method of claim 1 ; wherein the metric comprises an asynchronous time-stamped log entry, and wherein the method further comprises transforming asynchronous time-stamped log entries into one or more synchronous time series metrics.

26. A non-transitory computer readable medium having instructions stored thereon, execution of which by at least one processor in a computer system causes the computer system to perform a process of anomaly detection in an information technology system, the process comprising:

receiving an indication of a metric corresponding to a server in a system that includes a plurality of servers, the metric including data representative of a performance or operation of a machine in an information technology environment;

computing a statistic associated with the metric;

generating a plurality of feature vectors by generating, for each of the plurality of servers, a feature vector in n-dimensional space, each of the feature vectors including a metric for a corresponding server and including the statistic associated with the metric or a spectral characteristic of the metric for the corresponding server;

computing a center of mass of the plurality of feature vectors;

determining a degree of anomaly based on a distance of the server from the center of mass; and

detecting an anomaly associated with the server based on a result of said determining.

27. The non-transitory computer readable medium of claim 26 , wherein the metric includes a time-stamped log entry.

28. The non-transitory computer readable medium of claim 26 , further comprising:

generating an alert indicative of detection of the anomaly.

29. The non-transitory computer readable medium of claim 26 , further comprising:

generating an alert indicative of detection of the anomaly; and

directly triggering, in response to the alert, application of a protective measure to the server, by causing transmission of a message onto the network, wherein the message causes the server to be placed into a quarantine.

30. A system comprising:

a memory; and

at least one processor coupled to the memory and configured to perform operations including

receiving an indication of a metric corresponding to a server in a system that includes a plurality of servers, the metric including data representative of a performance or operation of a machine in an information technology environment;

computing a statistic associated with the metric;

generating a plurality of feature vectors by generating, for each of the plurality of servers, a feature vector in n-dimensional space, each of the feature vectors including a metric for a corresponding server and including the statistic associated with the metric or a spectral characteristic of the metric for the corresponding server;

computing a center of mass of the plurality of feature vectors;

determining a degree of anomaly based on a distance of the server from the center of mass; and

detecting an anomaly associated with the server based on a result of said determining.