Detecting global anomalies

1. A computer-implemented method for detecting global anomalies associated with a multi-node network of computing devices, the method comprising:

generating a reference global profile of a first event class at one or more nodes of the multi-node network, wherein generating the reference global profile of the first class event comprises:

determining a probability distribution for a reference event of the first event class, wherein the probability distribution is associated with a number of occurrences of the reference event of the first event class that occur over time at each node of the multi-node network;

identifying one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class, wherein the one or more groups of nodes include a portion of a total number of nodes in the multi-node network; and

generating a reference energy for each of the groups of nodes that correlate with the probability distribution for the reference event of the first event class;

generating a detection global profile of the first event class at one or more nodes of the multi-node network, wherein generating the detection global profile of the first class event comprises:

determining a probability distribution for a detection event of the first event class, wherein the probability distribution is associated with a number of occurrences of the detection event of the first event class that occur over time at each node in the one or more groups of nodes that have the correlation for the reference event of the first event class; and

generating a detection energy for each of the groups of nodes that correlate with the probability distribution for the reference event of the first event class;

comparing, at one or more nodes in the multi-node network, the reference energy with the detection energy for each of the groups of nodes having the correlation for the reference event of the first event class; and

determining if there is a deviation between the reference global profile and the detection global profile.

2. The method of claim 1 , wherein determining a probability distribution for the reference event of the first event class comprises:

selecting an epoch;

dividing the epoch into time segments; and

recording the reference event of the first event class occurring on each node in the multi-node network during the epoch.

3. The method of claim 2 , wherein the probability distribution for the reference event of the first event class is generated using a plurality of epochs.

4. The method of claim 1 , wherein identifying one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class comprises:

computing pair-wise correlation scores from the probability distribution for the reference event of the first event between each node on the multi-node network.

5. The method of claim 4 , wherein determining the probability distribution for the reference event of the first event class comprises computing a hash function that converts the first event class into a node address.

6. The method of claim 4 , wherein the pair-wise correlation scores comprise Pearson coefficients.

7. The method of claim 4 , wherein the reference energy (E) is calculated by

E

=

μ

1

+

σ

2

wherein μ is a mean of the absolute value of the pair-wise correlation scores between each node in a group of nodes that have a correlation and σ 2 is the variance.

8. The method of claim 1 , wherein generating a detection global profile comprises using a sliding window.

9. The method of claim 1 , further comprising:

generating a plurality of reference global profiles, each reference global profile comprising a different event class;

generating a plurality of detection global profiles, each detection global profile corresponding to a reference global profile to produce corresponding detection global profiles; and

determining if there is at least one deviation from at least one of the plurality of reference global profiles and the corresponding detection global profile.

10. A system for detecting global anomalies associated with a multi-node network of computing devices, the system comprising:

a multi-node network comprising a plurality of computing devices, wherein one or more of the computing devices on the multi-node network are configured to:

generate a reference global profile of a first event class, wherein generate the reference global profile of the first class event comprises:

determine a probability distribution for a reference event of the first event class, wherein the probability distribution is associated with a number of occurrences of the reference event of the first event class that occur over time at each node of the multi-node network;

identify one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class, wherein the one or more groups of nodes include a portion of a total number of nodes in the multi-node network; and

generate a reference energy for each of the groups of nodes that correlate with the probability distribution for the reference event of the first event class;

generate a detection global profile of a first event class, wherein generate the detection global profile of the first class event comprises:

determine a probability distribution for a detection event of the first event class, wherein the probability distribution is associated with a number of occurrences of the detection event of the first event class that occur over time at each node in the one or more groups of nodes that have the correlation for the reference event of the first event class; and

generate a detection energy for each of the groups of nodes that correlate with the probability distribution for the reference event of the first event class;

compare the reference energy with the detection energy for each of the groups of nodes having the correlation for the reference event of the first event class; and

determine if there is a deviation between the reference global profile and the detection global profile.

11. The system of claim 10 , wherein the one or more computing devices are further configured to compute a hash function for the first event class and the hash function converts the first event class into a node address.

12. The system of 10 , wherein the one or more computing devices are further configured to compute pair-wise correlations scores from the probability distribution for the reference event of the first event between each node on the multi-node network.

13. The system of 10 wherein the one or more computing devices are further configured to build a detection global profile using a sliding window.

14. The system of 10 , wherein the one or more computing devices are further configured to:

generate a plurality of reference global profiles, each reference global profile comprising a different event class;

generate a plurality of detection global profiles, each detection global profile corresponding to a reference global profile to produce corresponding detection global profiles; and

determine if there is at least one deviation from at least one of the plurality of reference global profiles and the corresponding detection global profile.

15. A system for detecting global anomalies comprising:

a multi-node network comprising a plurality of computing devices;

means for generating at least one reference global profile of a first event class, wherein generating at least one reference global profile of a first event class comprises:

means for determining a probability distribution for a reference event of the first event class, wherein the probability distribution is associated with a number of occurrences of the reference event of the first event class that occur over time at each node of the multi-node network;

means for identifying one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class, wherein the one or more groups of nodes include a portion of a total number of nodes in the multi-node network; and

means for generating a reference energy for each of the groups of nodes that correlate with the probability distribution for the reference event of the first event class;

means for generating at least one detection global profile of a first event class, wherein generating at least one detection global profile of a first even class comprises:

means for determining a probability distribution for a detection event of the first event class, wherein the probability distribution is associated with a number of occurrences of the detection event of the first event class that occur over time at each node in the one or more groups of nodes that have the correlation for the reference event of the first event class; and

means for generating a detection energy for each of the groups of nodes that correlate with the probability distribution for the reference event of the first event class;

means for comparing the reference energy with the detection energy for each of the groups of nodes having the correlation for the reference event of the first event class; and

means for determining if there is a deviation between the reference global profile and the detection global profile.

16. The system of claim 15 , wherein the means for generating at least one reference global profile of a first event class comprises means for recording reference events of the first event class on at least a portion of the nodes in the multi-node network.

17. The system of claim 16 , further comprising means for computing a hash function for the first event class and means for converting the hash function of the first event class into a node address.

18. The system of claim 15 , further comprising means for computing pair-wise correlations from the probability distribution for the reference event of the first event between each node on the multi-node network.

19. The system of claim 15 , further comprising means for generating a detection global profile using a sliding window.

20. The system of claim 15 , further comprising:

means for generating a plurality of reference global profiles, each reference global profile comprising a different event class;

means for generating a plurality of detection global profiles, each detection global profile corresponding to a reference global profile to produce a corresponding detection global profile; and

means for determining if there is at least one deviation from at least one of the plurality of reference global profiles and the corresponding detection global profile.

21. A non-transitory computer readable medium comprising:

computer executable instructions for detecting global anomalies in a multi-node network comprising one or more computing devices, the computer executable instructions configured to:

generate a reference global profile of a first event class, wherein generate the reference global profile of the first class event comprises:

determine a probability distribution for a reference event of the first event class, wherein the probability distribution is associated with a number of occurrences of the reference event of the first event class that occur over time at each node of the multi-node network;

identify one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class, wherein the one or more groups of nodes include a portion of a total number of nodes in the multi-node network; and

generate a reference energy for each of the groups of nodes that correlate with the probability distribution for the reference event of the first event class;

generate a detection global profile of a first event class, wherein generate the detection global profile of the first class event comprises:

determine a probability distribution for a detection event of the first event class, wherein the probability distribution is associated with a number of occurrences of the detection event of the first event class that occur over time at each node in the one or more groups of nodes that have the correlation for the reference event of the first event class; and

generate a detection energy for each of the groups of nodes that correlate with the probability distribution for the reference event of the first event class;

compare the reference energy with the detection energy for each of the groups of nodes having the correlation for the reference event of the first event; and

determine if there is a deviation between the reference global profile and the detection global profile.

22. The medium of claim 21 , wherein the computer executable instructions are further configured to record reference events of the first event class on each node of the multi-node network.

23. The medium of claim 21 , wherein the computer executable instructions are further configured to compute a hash function for the first event class and the hash function converts the first event class into a node address.

24. The medium of claim 21 , wherein the computer executable instructions are configured to compute pair-wise correlations from the probability distribution for the reference event of the first event between each node on the multi-node network.

25. The medium of claim 21 , wherein the computer executable instructions are further configured to build a detection global profile using a sliding window.

26. The medium of claim 21 , wherein the computer executable instructions are further configured to:

generate a plurality of reference global profiles, each reference global profile comprising a different event class;

generate a plurality of detection global profiles, each detection global profile corresponding to a reference global profile to produce corresponding detection global profiles; and

detect at least one deviation from at least one of the plurality of reference global profiles and the corresponding detection global profile.

27. The method of claim 1 , wherein the reference event of the first event class is a user login on the multi-node network.

28. The method of claim 1 , wherein each of the one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class are maximal cliques.

29. The method of claim 1 , further comprising flagging an anomaly if there is a deviation between the reference energy with the detection energy for one or more of the groups of nodes having the correlation for the reference event of the first event class.

30. The method of claim 1 , wherein identifying one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class comprises identifying pairs of nodes having a correlation score above a threshold value.

31. The method of claim 1 , wherein determining a probability distribution for a reference event of the first event class comprises recording the reference event on each of the nodes in the multi-node network.

32. The system of claim 10 , wherein the reference event of the first event class is a user login on the multi-node network.

33. The system of claim 10 , wherein each of the one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class are maximal cliques.

34. The system of claim 10 , wherein the one or more computing devices is further configured to flag an anomaly if there is a deviation between the reference energy with the detection energy for one or more of the groups of nodes having the correlation for the reference event of the first event class.

35. The system of claim 10 , wherein identify one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class comprises identify pairs of nodes having a correlation score above a threshold value.

36. The system of claim 10 , wherein determining a probability distribution for a reference event of the first event class comprises recording the reference event on each of the nodes in the multi-node network.

37. The system of claim 12 , wherein the reference energy (E) is calculated by

E

=

μ

1

+

σ

2

wherein μ is a mean of the absolute value of the pair-wise correlation scores between each node in a group of nodes that have a correlation and σ 2 is the variance.

38. The system of claim 12 , wherein the pair-wise correlation scores comprise Pearson coefficients.

39. The system of claim 10 , wherein generating the probability distribution for the reference event of the first event class comprises:

select an epoch;

divide the epoch into time segments; and

record the reference event of the first event class occurring on each node in the multi-node network during the epoch.

40. The system of claim 39 , wherein the probability distribution for the reference event of the first event class is generated using a plurality of epochs.

41. The system of claim 15 , wherein the reference event of the first event class is a user login on the multi-node network.

42. The system of claim 15 , wherein each of the one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class are maximal cliques.

43. The system of claim 15 , further comprising means for flagging an anomaly if there is a deviation between the reference energy with the detection energy for one or more of the groups of nodes having the correlation for the reference event of the first event class.

44. The system of claim 15 , wherein identifying one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class comprises means for identifying pairs of nodes having a correlation score above a threshold value.

45. The system of claim 15 , wherein means for determining a probability distribution for a reference event of the first event class comprises means for recording the reference event on each of the nodes in the multi-node network.

46. The system of claim 18 , wherein the reference energy (E) is calculated by

E

=

μ

1

+

σ

2

wherein μ is a mean of the absolute value of the pair-wise correlation scores between each node in a group of nodes that have a correlation and σ 2 is the variance.

47. The system of claim 18 , wherein the pair-wise correlation scores comprise Pearson coefficients.

48. The system of claim 15 , wherein the means for generating the probability distribution for the reference event of the first event class comprises:

means for selecting an epoch;

means for dividing the epoch into time segments; and

means for recording the reference event of the first event class occurring on each node in the multi-node network during the epoch.

49. The system of claim 48 , wherein the probability distribution for the reference event of the first event class is generated using a plurality of epochs.

50. The computer-readable medium of claim 21 , wherein the reference event of the first event class is a user login on the multi-node network.

51. The computer-readable medium of claim 21 , wherein each of the one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class are maximal cliques.

52. The computer-readable medium of claim 21 , wherein the computer executable instructions are further configured to flag an anomaly if there is a deviation between the reference energy with the detection energy for one or more of the groups of nodes having the correlation for the reference event of the first event class.

53. The computer-readable medium of claim 21 , wherein identify one or more groups of nodes that have a correlation with the probability distribution for the reference event of the first event class comprises identify pairs of nodes having a correlation score above a threshold value.

54. The computer-readable medium of claim 21 , wherein determining a probability distribution for a reference event of the first event class comprises recording the reference event on each of the nodes in the multi-node network.

55. The computer-readable medium of claim 24 , wherein the reference energy (E) is calculated by

E

=

μ

1

+

σ

2

wherein μ is a mean of the absolute value of the pair-wise correlation scores between each node in a group of nodes that have a correlation and σ 2 is the variance.

56. The computer-readable medium of claim 24 , wherein the pair-wise correlation scores comprise Pearson coefficients.

57. The computer-readable medium of claim 24 , wherein generating the probability distribution for the reference event of the first event class comprises:

select an epoch;

divide the epoch into time segments; and

record the reference event of the first event class occurring on each node in the multi-node network during the epoch.