Automatic triaging of diagnostics failures
Non-limiting examples of systems, methods, and devices for automatic triaging of diagnostic failures for heterogeneous groups of tenants of a Software-as-a-Service, multi-tenant environment are disclosed herein. In an implementation, telemetry data for the heterogeneous groups of tenants is analyzed to classify individual tenant failures and detect the health status of the individual tenant. Tenant failures and/or tenant health statuses are filtered according to a threshold level. Anomalies having metrics that meet or exceed the threshold level are further analyzed to determine their priority (e.g., to a specific tenant). If the anomalies are known, then an existing entry for the anomaly is tagged and its priority may be changed. If the anomalies are unknown, then an entry is generated for the anomaly and prioritized. Tenants may be notified of a detected anomaly and may provide feedback. The feedback may be used to update triaging models.
1. A computing device comprising:
a memory for storing executable program code;
a processor functionally coupled to the memory; and
the executable program code that, when executed by the computing device, directs the processor to:
determine a health classification for a tenant, in a multi-tenant environment, based on diagnostic information associated with the tenant;
receive telemetry data for the tenant;
identify anomalies in the telemetry data received for the tenant;
determine a classification for one or more of the anomalies based on the telemetry data; and
for at least an anomaly of the one or more of the anomalies:
determine a severity score for the anomaly based at least on the classification determined for the anomaly and the health classification determined for the tenant; and
determine an impact of the anomaly to the tenant based at least on the severity score determined for the anomaly;
generate triage data for the anomaly;
identify an existing mitigation effort applicable to the anomaly and associated with a user distinguished from tenants in the multi-tenant environment;
generate an updated mitigation effort based on the triage data; and
send a notification comprising details about the updated mitigation effort.
2. The computing device of claim 1 , wherein the computing device comprises a machine learning environment.
3. The computing device of claim 2 , wherein the machine learning environment comprises an anomaly classifier configured to:
determine classifications for the anomalies in the telemetry data, including the classification for the anomaly; and
generate filtered telemetry data based on the classifications of the anomalies in the telemetry data.
4. The computing device of claim 3 , wherein the machine learning environment comprises a tenant health classifier configured to:
determine health classifications for a plurality of tenants, including the health classification for the tenant; and
generate filtered diagnostics data based on the health classifications.
5. The computing device of claim 1 , wherein to identify the existing mitigation effort applicable to the anomaly, the executable program code directs the processor to identify an entry in a repository of mitigation efforts.
6. The computing device of claim 1 , wherein the executable program code further directs the processor to generate the notification for transmission to a tenant in the multi-tenant environment.
7. The computing device of claim 1 , wherein to determine the impact of the anomaly to the tenant based at least on the severity score, the executable program code directs the processor to determine that the impact is enterprise blocking.
8. A computer-implemented method, comprising:
determining a health classification for a tenant, in a multi-tenant environment, based on diagnostic information associated with the tenant;
receiving telemetry data for the tenant;
identifying anomalies in the telemetry data received for the tenant;
determining a classification for one or more of the anomalies based on the telemetry data; and
for at least an anomaly of the one or more of the anomalies:
determining a severity score for the anomaly based at least on the classification determined for the anomaly and the health classification determined for the tenant;
determining an impact of the anomaly to the tenant based at least on the severity score determined for the anomaly;
generating triage data for the anomaly;
identifying an existing mitigation effort applicable to the anomaly and associated with a user distinguished from tenants in the multi-tenant environment;
generating an updated mitigation effort based on the triage data; and
sending a notification comprising details about the updated mitigation effort.
9. The computer-implemented method of claim 8 , further comprising:
determining classifications for the anomalies in the telemetry data, including the classification for the anomaly;
generating filtered telemetry data based on the classifications of the anomalies in the telemetry data.
10. The method of claim 9 , wherein the method further comprises: determining health classifications for a plurality of tenants, including the health classification for the tenant; and generating filtered diagnostics data based on the health classifications.
11. The computer-implemented method of claim 8 , wherein identifying the existing mitigation effort comprises identifying an entry in a repository of mitigation efforts.
12. The computer-implemented method of claim 8 , further comprising generating the notification for transmission to a tenant in the multi-tenant environment.
13. The method of claim 8 , wherein determining the impact of the anomaly to the tenant based at least on the severity score comprises determining that the impact is enterprise blocking.
14. The method of claim 13 , further comprising generating the triage data based on the impact to the tenant.
15. A computer readable storage device comprising executable instructions that, when executed by a processor, cause the processor to:
determine a health classification for a tenant, in a multi-tenant environment, based on diagnostic information associated with the tenant;
receive telemetry data for the tenant;
identify anomalies in the telemetry data received for the tenant;
determine a classification for one or more of the anomalies based on the telemetry data; and
for at least an anomaly of the one or more of the anomalies:
determine a severity score for the anomaly based at least on the classification determined for the anomaly and the health classification determined for the tenant;
determine an impact of the anomaly to the tenant based at least on the severity score determined for the anomaly
generate triage data for the anomaly;
identify an existing mitigation effort applicable to the anomaly and associated with a user distinguished from tenants in the multi-tenant environment;
generate an updated mitigation effort based on the triage data; and
send a notification comprising details about the updated mitigation effort.
16. The computer readable storage device of claim 15 , wherein the executable instructions further cause the processor to: determine classifications for the anomalies in the telemetry data, including the classification for the anomaly; generate filtered telemetry data based on the classifications of the anomalies in the telemetry data.
17. The computer readable storage device of claim 16 , wherein the executable instructions further cause the processor to:
determine health classifications for a plurality of tenants, including the health classification for the tenant; and
generate filtered diagnostics data based on the health classifications.
18. The computer readable storage device of claim 15 , wherein to identify the existing mitigation effort, the executable instructions cause the processor to identify an entry in a repository of mitigation efforts.
19. The computer readable storage device of claim 15 , wherein the executable instructions further cause the processor to generate the notification for transmission to an anomaly mitigation team member.
20. The computer readable storage device of claim 15 , wherein to determine the impact of the anomaly to the tenant based at least the severity score, the executable instructions cause the processor to determine that the impact is enterprise blocking.