IP Library › Granted Patent US 11,030,214
Granted Patent B2
US 11,030,214 · App. 16/391,188 · Granted Jun 8, 2021

System for identification of outlier groups

Inventors: Aleksey Ashikhmin (Redmond, WA); Sanjay H Ramaswamy (Redmond, WA); Andrew C. Owen (Redmond, WA); Aaron Harrison (Redmond, WA); Sreeram Nivarthi (Redmond, WA); Vindana Madhuwantha (Redmond, WA); Umashree Narayanaswamy (Redmond, WA); Brian Quistorff (Redmond, WA); Eric Radtke (Redmond, WA); Omar Mustafa (Redmond, WA); Di Li (Redmond, WA)
Assignee: Microsoft Technology Licensing, LLC
G06F16/26G06F9/451G06F16/2465G06F2216/03
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,030,214
App. No.
16/391,188
Granted
Jun 8, 2021
Kind
B2
Abstract

A method may include retrieving metric data on a plurality of groups of users, the metric data including: a value of a performance metric for each of the plurality of groups; and an indication that a first group of the plurality of groups is anomalous with respect to a value of the performance metric of a control group of the plurality of groups; and presenting a user interface, the user interface including: a first portion including a visualization of a comparison of the value of the performance metric for the first group and values of the performance metric of other groups in the plurality of groups; and a second portion including a visualization of trend data of the performance metric for the first group over a period of time.

Claims (56)

1. A system comprising:

a processing unit; and

a storage device comprising instructions, which when executed by the processing unit, configure the processing unit to:

retrieve metric data on a plurality of groups of users, the metric data including a value of a performance metric for each of the plurality of groups;

determine a control group of the plurality of groups utilizing a model based on a distance metric to calculate alikeness between users;

determine that a first group of the plurality of groups is anomalous with respect to a value of the performance metric of the control group, wherein determining that the first group is anomalous includes determining that a difference, between a median value of the performance metric for the plurality of users and a median value of the performance metric for the control group, exceeds a threshold value; and

present a user interface, the user interface including:

a first portion including a visualization of a comparison of the value of the performance metric for the first group and values of the performance metric of other groups in the plurality of groups, and an indication of the determined anomaly with reference to the determined control group; and

a second portion including a visualization of trend data of the performance metric for the first group over a period of time.

2. The system of claim 1 , wherein each of the users in the plurality of users is associated with attribute values for a plurality of attributes.

3. The system of claim 2 , wherein the users in the plurality of groups are selected based on sharing a value of a selected attribute of the plurality of attributes.

4. The system of claim 3 , wherein users in the control group are selected based on attributes other than the selected attribute.

5. The system of claim 3 , wherein the selected attribute is a reporting attribute with respect to an organization.

6. The system of claim 1 , wherein the user interface further includes a selection element configured to change the performance metric, wherein determining the control group includes selecting data items of the metric data that influence the performance metric.

7. The system of claim 6 , wherein the instructions, which when executed by the processing unit, configure the processing unit to:

receive selection of a new metric via the selection element;

access further metric data on the plurality of groups of users, the further metric data including:

a value for the new metric for each of the plurality of groups; and

an indication that a second group of the plurality of groups is anomalous with respect to a control group of the plurality of groups for the new metric; and

update the user interface based on the further metric data.

8. The system of claim 1 , wherein the instructions, which when executed by the processing unit, configure the processing unit to:

receive a selection in the first portion of a second group of the plurality of groups; and

in response to the selection, update the second portion to include a visualization of trend data of the performance metric for the second group.

9. A storage device comprising instructions, which when executed by a processing unit, configure the processing unit to:

retrieve metric data on a plurality of groups of users, the metric data including a value of a performance metric for each of the plurality of groups;

determine a control group of the plurality of groups utilizing a model based on a distance metric to calculate alikeness between users;

determine that a first group of the plurality of groups is anomalous with respect to a value of the performance metric of the control group, wherein determining that the first group is anomalous includes determining that a difference, between a median value of the performance metric for the plurality of users and a median value of the performance metric for the control group, exceeds a threshold value; and

present a user interface, the user interface including:

a first portion including a visualization of a comparison of the value of the performance metric for the first group and values of the performance metric of other groups in the plurality of groups, and an indication of the determined anomaly with reference to the determined control group; and

a second portion including a visualization of trend data of the performance metric for the first group over a period of time.

10. The storage device of claim 9 , wherein each of the users in the plurality of users is associated with attribute values for a plurality of attributes.

11. The storage device of claim 10 , wherein the users in the plurality of groups are selected based on sharing a value of a selected attribute of the plurality of attributes.

12. The storage device of claim 11 , wherein users in the control group are selected based attributes other than the selected attribute.

13. The storage device of claim 11 , wherein the selected attribute is a reporting attribute with respect to an organization.

14. The storage device of claim 9 , wherein the user interface further includes a selection element configured to change the performance metric, wherein determining the control group includes selecting data items of the metric data that influence the performance metric.

15. The storage device of claim 9 , wherein the instructions, which when executed by the processing unit, configure the processing unit to:

receive selection of a new metric via the selection element;

access further metric data on the plurality of groups of users, the further metric data including:

a value for the new metric for each of the plurality of groups; and

an indication that a second group of the plurality of groups is anomalous with respect to a control group of the plurality of groups for the new metric; and

update the user interface based on the further metric data.

16. The storage device of claim 9 , wherein the instructions, which when executed by the processing unit, configure the processing unit to:

receive a selection in the first portion of a second group of the plurality of groups; and

in response to the selection, update the second portion to include a visualization of trend data of the performance metric for the second group.

17. A method comprising:

retrieving metric data on a plurality of groups of users, the metric data including a value of a performance metric for each of the plurality of groups;

determining a control group of the plurality of groups utilizing a model based on a distance metric to calculate alikeness between users;

determining that a first group of the plurality of groups is anomalous with respect to a value of the performance metric of the control group, wherein determining that the first group is anomalous includes determining that a difference, between a median value of the performance metric for the plurality of users and a median value of the performance metric for the control group, exceeds a threshold value; and

presenting a user interface, the user interface including:

a first portion including a visualization of a comparison of the value of the performance metric for the first group and values of the performance metric of other groups in the plurality of groups, and an indication of the determined anomaly with reference to the determined control group; and

a second portion including a visualization of trend data of the performance metric for the first group over a period of time.

18. The method of claim 17 , wherein each of the users in the plurality of users is associated with attribute values for a plurality of attributes.

19. The system of claim 1 , wherein the instructions, which when executed by the processing unit, configure the processing unit to:

determine a selection of a first dimension from a plurality of dimensions of the metric data; and

form the first group to include users having a same value of the first dimension.

20. The storage device of claim 9 , wherein determining that the first group is anomalous with respect to the control group of the plurality of groups is based on a difference between a median value of the performance metric for the plurality of users and a median value of the performance metric for the control group exceeding a threshold.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2019
From: ASHIKHMIN, ALEKSEY; RAMASWAMY, SANJAY H; OWEN, ANDREW C.; HARRISON, AARON; NIVARTHI, SREERAM; MADHUWANTHA, VINDANA; NARAYANASWAMY, UMASHREE; QUISTORFF, BRIAN; RADTKE, ERIC; MUSTAFA, OMAR; LI, DI
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 049640/0030 →
Continuity (1)
Related Publication 20200334273A1 · Oct 22, 2020