Anomaly detection in performance management
Methods and systems for detecting anomalous behavior include performing a principal component analysis on a plurality of key performance indicators (KPIs) to determine a set of principal axes. The KPIs are clustered in a space defined by the set of principal axes. Local and structural anomalies are determined in the clustered KPIs. The structural and local anomalies are classified based on historical information. A management action is performed based on the classified structural and local anomalies.
1. A method for detecting anomalous behavior, comprising:
performing a principal component analysis on a plurality of key performance indicators (KPIs) to determine a set of principal axes;
clustering the KPIs in a space defined by the set of principal axes;
determining structural anomalies in the clustered KPIs based on whether a cluster of KPIs that does not conform to a Gaussian distribution had conformed to a Gaussian distribution at a previous time;
determining local anomalies in the clustered KPIs;
classifying the structural and local anomalies based on historical information; and
performing a management action based on the classified structural and local anomalies.
2. The method of claim 1 , wherein determining structural anomalies comprises determining how closely each cluster of KPIs conforms to a Gaussian distribution.
3. The method of claim 1 , wherein determining local anomalies comprises comparing a distance of each individual KPI from its respective cluster mean to a threshold.
4. The method of claim 3 , wherein determining local anomalies comprises performing said comparison only for KPIs in clusters that conform to a Gaussian distribution.
5. The method of claim 3 , wherein classifying local anomalies comprises determining whether a KPI is an isolated outlier.
6. The method of claim 5 , wherein classifying local anomalies comprises determining whether the KPI has gone from a distance below a first threshold at a previous sampling time to a current distance that is above a second, higher threshold.
7. The method of claim 5 , wherein classifying local anomalies comprises determining whether there is a trend across multiple previous sampling times from a distance below a first threshold toward a second, higher threshold.
8. The method of claim 1 , further comprising transforming from a space based on the principal axes to an original space and determining whether each of the local and structural anomalies is a global anomaly or a local anomaly.
9. A non-transitory computer readable storage medium comprising a computer readable program for detecting anomalous behavior, wherein the computer readable program when executed on a computer causes the computer to perform the steps of:
performing a principal component analysis on a plurality of key performance indicators (KPIs) to determine a set of principal axes;
clustering the KPIs in a space defined by the set of principal axes;
determining structural anomalies in the clustered KPIs based on whether a cluster of KPIs that does not conform to a Gaussian distribution had conformed to a Gaussian distribution at a previous time;
determining local anomalies in the clustered KPIs;
classifying the structural and local anomalies based on historical information; and
performing a management action based on the classified structural and local anomalies.
10. A system for detecting anomalous behavior, comprising:
a detection module comprising a processor configured to perform a principal component analysis on a plurality of key performance indicators (KPIs) to determine a set of principal axes, to cluster the KPIs in a space defined by the set of principal axes, to determine structural anomalies in the clustered KPIs based on whether a cluster of KPIs that does not conform to a Gaussian distribution had conformed to a Gaussian distribution at a previous time, and to determine local anomalies in the clustered KPIs;
a classification module configured to classify the structural and local anomalies based on historical information; and
a management module configured to perform a management action based on the classified structural and local anomalies.
11. The system of claim 10 , wherein the detection module is further configured to determine how closely each cluster of KPIs conforms to a Gaussian distribution.
12. The system of claim 10 , wherein the detection module is further configured to compare a distance of each individual KPI from its respective cluster mean to a threshold.
13. The system of claim 12 , wherein the classification module is further configured to perform said comparison only for KPIs in clusters that conform to a Gaussian distribution.
14. The system of claim 12 , wherein the classification module is further configured to determine whether a KPI is an isolated outlier.
15. The system of claim 14 , wherein the classification module is further configured to determine whether the KPI has gone from a distance below a first threshold at a previous sampling time to a current distance that is above a second, higher threshold.
16. The system of claim 14 , wherein the classification module is further configured to determine whether there is a trend across multiple previous sampling times from a distance below a first threshold toward a second, higher threshold.
17. The system of claim 10 , wherein the classification module is further configured to transform from a space based on the principal axes to an original space and to determine whether each of the local and structural anomalies is a global anomaly or a local anomaly.