IP Library › Granted Patent US 11,601,325
Granted Patent B2
US 11,601,325 · App. 17/390,411 · Granted Mar 7, 2023

Method and system for evaluating peer groups for comparative anomaly

Inventors: Rachel Lemberg (Herzliya, IL); Yaniv Lavi (Herzliya, IL); Dor Bank (Herzliya, IL); Raphael Fettaya (Herzliya, IL)
Assignee: Microsoft Technology Licensing, LLC
H04L41/065H04L67/75
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,601,325
App. No.
17/390,411
Filed
Jul 30, 2021
Granted
Mar 7, 2023
Kind
B2
Art Unit
2444
USPC
709/224
Abstract

Example aspects include techniques for implementing peer group evaluation for comparative anomaly. These techniques may include determining a candidate group including a plurality of component metrics, and determining that the plurality of component metrics are a peer group based at least in part on a cluster profile of the candidate group and the candidate group exhibiting peer-like behavior of a period of time. In addition, the techniques may include detecting anomalous activity based at least in part on first performance information of a component metric deviating from second performance information for the peer group, and providing a notification of the anomalous activity.

Claims (75)

1. A cloud computing device, comprising:

a memory storing instructions; and

at least one processor communicatively coupled with the memory and configured to:

determine a candidate group including a plurality of component metrics, each component metric corresponding to an associated cloud component and expected to have measurements within a range associated with the candidate group;

determine that the plurality of component metrics are a peer group based at least in part on a cluster profile of the candidate group and the candidate group exhibiting peer-like behavior over a period of time, wherein a clustering algorithm is used to generate the cluster profile from historical metrics information of the candidate group, the cluster profile indicating a number of persistent clusters associated with the candidate group, the number of persistent clusters being determined to be less than a preconfigured threshold;

collect first performance information for a component metric of the plurality of component metrics;

detect anomalous activity based at least in part on the first performance information deviating from second performance information for the peer group; and

provide a notification of the anomalous activity.

2. The cloud computing device of claim 1 , wherein to determine the candidate group, the at least one processor is configured to:

receive, via a graphical user interface (GUI), user input information identifying the plurality of component metrics.

3. The cloud computing device of claim 1 , wherein to determine the candidate group, the at least one processor is configured to:

identify the plurality of component metrics based at least in part on performance information and/or a shared attribute.

4. The cloud computing device of claim 1 , wherein the candidate group is a first candidate group, the plurality of component metrics is a first plurality of component metrics, the peer group is a first peer group, the period of time is a first period of time, and the at least one processor is further configured to:

determine a second candidate group including a second plurality of component metrics, each component metric of the second plurality of component metrics expected to have a peer-like performance within the second candidate group;

determine that a third plurality of component metrics are a second peer group based at least in part on a cluster profile of the second candidate group and the third plurality of component metrics exhibiting peer-like behavior over a second period of time, wherein the third plurality of component metrics is a subset of the second plurality of component metrics and a first total number of component metrics of the second plurality of component metrics is greater than a second total number of component metrics of the third plurality of component metrics; and

perform anomaly detection based on the second peer group.

5. The cloud computing device of claim 1 , wherein the component metric is a first component metric, and the at least one processor is further configured to:

detect a deviation by a second component metric of the plurality of component metrics from the peer group; and

remove the second component metric from the peer group based on the deviation.

6. The cloud computing device of claim 1 , wherein to determine that the plurality of component metrics are the peer group, the at least one processor is configured to:

determine an upper bound and a lower bound of an adjustable boxplot;

generate a plurality of time series based on historical metrics information of the candidate group;

determine that the plurality of time series do not include a first time series having a first percentage of points outside of the adjustable boxplot that is greater than a first preconfigured threshold; and

determine that the plurality of time series do not include a second time series having a second percentage of consecutive points outside of the adjustable boxplot that is greater than a second preconfigured threshold.

7. The cloud computing device of claim 6 , wherein to determine the upper bound and the lower bound of the adjustable boxplot, the at least one processor is configured to:

determine two highest values with a first largest difference;

select a lower value of the two highest values as an upper bound of the adjustable boxplot;

determine two lowest values with a second largest difference; and

select a higher value of the two lowest values as the lower bound of the adjustable boxplot.

8. The cloud computing device of claim 1 , wherein to detect the anomalous activity, the at least one processor is configured to:

generate a time series slice based on the first performance information, the second performance information, and an adjustable boxplot;

determine a first number of points of the first performance information outside the adjustable boxplot;

determine a first percentage of points of the first performance information outside the adjustable boxplot;

determine a second number of consecutive points of the first performance information outside the adjustable boxplot;

determine a second percentage of consecutive points of the first performance information outside the adjustable boxplot; and

detect the anomalous activity based on the first number, the first percentage, the second number, or the second percentage being above a preconfigured threshold.

9. The cloud computing device of claim 1 , wherein each component metric of the plurality of component metrics corresponds to a measurement of an attribute of a cloud application, a cloud service, and/or a cloud resource.

10. The cloud computing device of claim 1 , wherein each component metric of the plurality of component metrics measures one of a request duration, a dependency duration, or client performance.

11. A method, comprising:

determining a candidate group including a plurality of component metrics, each component metric corresponding to an associated cloud component and expected to have measurements within a range associated with the candidate group;

determining that the plurality of component metrics are a peer group based at least in part on a cluster profile of the candidate group and the candidate group exhibiting peer-like behavior over a period of time, wherein a clustering algorithm is used to generate the cluster profile from historical metrics information of the candidate group, the cluster profile indicating a number of persistent clusters associated with the candidate group, the number of persistent clusters being determined to be less than a preconfigured threshold;

collecting first performance information for a component metric of the plurality of component metrics;

detecting anomalous activity based at least in part on the first performance information deviating from second performance information for the peer group; and

providing a notification of the anomalous activity.

12. The method of claim 11 , wherein determining that the plurality of component metrics are the peer group, comprises:

determining an upper bound and a lower bound of an adjustable boxplot;

generating a plurality of time series based on historical metrics information of the candidate group;

determining that the plurality of time series do not include a first time series having a first percentage of points outside of the adjustable boxplot that is greater than a first preconfigured threshold; and

determining that the plurality of time series do not include a second time series having a second percentage of consecutive points outside of the adjustable boxplot that is greater than a second preconfigured threshold.

13. The method of claim 11 , wherein detecting the anomalous activity, comprises:

generating a time series slice based on the first performance information, the second performance information, and an adjustable boxplot;

determining a first number of points of the first performance information outside the adjustable boxplot;

determining a first percentage of points of the first performance information outside the adjustable boxplot;

determining a second number of consecutive points of the first performance information outside the adjustable boxplot;

determining a second percentage of consecutive points of the first performance information outside the adjustable boxplot; and

detecting the anomalous activity based on the first number, the first percentage, the second number, or the second percentage being above a preconfigured threshold.

14. The method of claim 11 , wherein each component metric of the plurality of component metrics corresponds to a measurement of an attribute of a cloud application, a cloud service, and/or a cloud resource.

15. The method of claim 11 , wherein each component metric of the plurality of component metrics measures one of a request duration, a dependency duration, or client performance.

16. A non-transitory computer-readable device storing instructions thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:

determining a candidate group including a plurality of component metrics, each component metric corresponding to an associated cloud component and expected to have measurements within a range associated with the candidate group;

determining that the plurality of component metrics are a peer group based at least in part on a cluster profile of the candidate group and the candidate group exhibiting peer-like behavior of a period of time, wherein a clustering algorithm is used to generate the cluster profile from historical metrics information of the candidate group, the cluster profile indicating a number of persistent clusters associated with the candidate group, the number of persistent clusters being determined to be less than a preconfigured threshold;

collecting first performance information for a first component metric of the plurality of component metrics;

detecting anomalous activity based at least in part on the first performance information deviating from second performance information for the peer group; and

providing a notification of the anomalous activity.

17. The non-transitory computer-readable device of claim 16 , wherein determining that the plurality of component metrics are the peer group, comprises:

determining an upper bound and a lower bound of an adjustable boxplot;

generating a plurality of time series based on historical metrics information of the candidate group;

determining that the plurality of time series do not include a first time series having a first percentage of points outside of the adjustable boxplot that is greater than a first preconfigured threshold; and

determining that the plurality of time series do not include a second time series having a second percentage of consecutive points outside of the adjustable boxplot that is greater than a second preconfigured threshold.

18. The non-transitory computer-readable device of claim 16 , wherein determining the candidate group comprises:

identifying one or more of the plurality of component metrics based on at least one of performance information or a shared attribute.

19. The non-transitory computer-readable device of claim 16 , the operations further comprising:

detecting a deviation by a second component metric of the plurality of component metrics from the peer group; and

removing the second component metric from the peer group based on the deviation.

20. The non-transitory computer-readable device of claim 16 , wherein each component metric of the plurality of component metrics measures one of a request duration, a dependency duration, or client performance.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 15, 2022
From: FETTAYA, RAPHAEL
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 059011/0371 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 30, 2021
From: LEMBERG, RACHEL; LAVI, YANIV; BANK, DOR
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 057039/0238 →
Continuity (1)
Related Publication 20230033647A1 · Feb 2, 2023