Health model for cloud service health monitoring
The techniques described herein automatically correlate the health of cloud resources to a broader health determination for an entity executing within, or supported by, a distributed computing environment. In contrast to the typical manual analysis that is required to make a broader health determination for a specific entity, the techniques generate and use a standard health model that can be applied, or scaled, to detect unhealthy scenarios across a variety of different entities with different owners (e.g., different tenants and/or different cloud resource providers). Furthermore, to meet varying owner perspectives on health, the techniques include a layer on top of the standard health model that enables an owner to provide input that customizes the standard health model for their own entity.
1. A method comprising:
generating a directed graph health model that defines dependencies between nodes within a distributed computing environment, wherein:
the nodes include lower-level nodes representing lower-level entities;
the nodes include higher-level nodes representing higher-level entities; and
edges that connect respective pairs of nodes, the edges representing dependencies between the respective pairs of nodes;
for an individual lower-level entity of the lower-level entities:
monitoring, by the directed graph health model, values of a plurality of metrics that are collected in association with use of the individual lower-level entity; and
categorizing, by the directed graph health model, the individual lower-level entity as one of healthy or unhealthy by applying an anomaly detection algorithm to the values of the plurality of metrics;
determining, by the directed graph health model, a number of lower-level entities have been categorized as unhealthy, wherein each lower-level entity in the number of lower-level entities is connected to a higher-level entity in the directed graph health model via a respective edge;
determining, by the directed graph health model, that the number of lower-level entities satisfies a threshold established to indicate that a state of the higher-level entity is unhealthy, wherein the threshold defines a percentage of a total number of lower-level entities connected to the higher-level entity in the directed graph health model;
in response to determining that the number of lower-level entities satisfies the threshold, accessing, by the directed graph health model, a rule associated with the higher-level entity, the rule defining an action to execute for the higher-level entity when the state of the higher-level entity is unhealthy; and
executing, by the directed graph health model, the action for the higher-level entity.
2. The method of claim 1 , wherein the action comprises one of:
providing, to an owner of the higher-level entity, a notification indicating that the state of the higher-level entity is unhealthy;
transitioning the state of the higher-level entity from unhealthy to healthy by allocating cloud resources to the higher-level entity; or
transitioning the state of the higher-level entity from unhealthy to healthy by implementing a set of mitigation measures on the number of lower-level entities entity, wherein the set of mitigation measures is defined by the rule.
3. The method of claim 1 , wherein each lower-level entity of the lower-level entities and each higher-level entity of the higher-level entities comprises an identification parameter to distinguish one entity from a next entity.
4. The method of claim 3 , wherein the higher-level entities are different types of higher-level entities, the different types of higher-level entities comprising a tenant service type higher-level entity, a cloud resource provider service type higher-level entity, a geographic region type higher-level entity, a tenant type higher-level entity, or a cloud resource provider type higher-level entity.
5. The method of claim 4 , wherein the threshold established to indicate when the state of the higher-level entity is unhealthy is established based on a type of the higher-level entity.
6. The method of claim 3 , wherein the lower-level entities are of different types of lower-level entities, the different types of lower-level entities comprising a virtual machine type lower-level entity, a storage unit type lower-level entity, a container type lower-level entity, a physical server type lower-level entity, a network switch type lower-level entity, a container registry type lower-level entity, a key vault instance type lower-level entity, or a micro-service type lower-level entity.
7. The method of claim 3 , further comprising:
causing the directed graph health model to be displayed on a display device along with a graphical indication that the state of the higher-level entity is unhealthy;
receiving a user selection of the higher-level entity via the directed graph health model caused to be displayed on the display device; and
based on the user selection, causing the identification parameters associated with the number of lower-level entities that have been categorized as unhealthy to be displayed.
8. The method of claim 7 , further comprising:
receiving another user selection of another higher-level entity via the directed graph health model caused to be displayed on the display device, wherein the higher-level entity is connected to the other higher-level entity via a respective edge in the directed graph health model; and
causing an update to the identification parameters associated with the number of lower-level entities that have been categorized as unhealthy to be displayed, wherein the update reduces the number of lower-level entities that have been categorized as unhealthy to those that are connected to both the higher-level entity and the other higher-level entity in the directed graph health model.
9. The method of claim 1 , wherein the percentage and the total number of lower-level entities are specific to a type of lower-level entity.
10. The method of claim 1 , wherein the threshold is determined by a machine learning model configured to predict when a performance of the higher-level entity is degraded to a minimum threshold performance.
11. The method of claim 1 , wherein the higher-level entity, the threshold, and the rule are defined based on input from a tenant of the distributed computing environment, wherein the tenant owns the higher-level entity.
12. A system comprising:
a processing system; and
a computer readable storage medium storing instructions that, when executed by the processing system, cause the system to perform operations comprising:
generating a directed graph health model that defines dependencies between nodes within a distributed computing environment, wherein:
the nodes include lower-level nodes representing lower-level entities;
the nodes include higher-level nodes representing higher-level entities; and
edges that connect respective pairs of nodes, the edges representing dependencies between the respective pairs of nodes;
for an individual lower-level entity of the lower-level entities:
monitoring, via the directed graph health model, values of a plurality of metrics that are collected in association with use of the individual lower-level entity; and
categorizing, via the directed graph health model, the individual lower-level entity as one of healthy or unhealthy by applying an anomaly detection algorithm to the values of the plurality of metrics;
determining, via the directed graph health model, a number of lower-level entities have been categorized as unhealthy, wherein each lower-level entity in the number of lower-level entities is connected to a higher-level entity in the directed graph health model via a respective edge;
determining, via the directed graph health model, that the number of lower-level entities satisfies a threshold established to indicate that a state of the higher-level entity is unhealthy, wherein the threshold defines a percentage of a total number of lower-level entities connected to the higher-level entity in the directed graph health mode;
in response to determining that the number of lower-level entities satisfies the threshold, accessing, via the directed graph health model, a rule associated with the higher-level entity, the rule defining an action to execute for the higher-level entity when the state of the higher-level entity is unhealthy; and
executing, via the directed graph health model, the action for the higher-level entity.
13. The system of claim 12 , wherein the operations further comprise:
causing the directed graph health model to be displayed on a display device along with a graphical indication that the state of the higher-level entity is unhealthy;
receiving a user selection of the higher-level entity via the directed graph health model caused to be displayed on the display device; and
based on the user selection, causing the identification parameters associated with the number of lower-level entities that have been categorized as unhealthy to be displayed.
14. The system of claim 12 , wherein the higher-level entity, the threshold, and the rule are defined based on input from a tenant of the distributed computing environment, wherein the tenant owns the higher-level entity.
15. A method comprising:
generating a directed graph health model that defines dependencies between nodes within a distributed computing environment, wherein:
the nodes include lower-level nodes representing lower-level entities;
the nodes include higher-level nodes representing higher-level entities; and
edges that connect respective pairs of nodes, the edges representing dependencies between the respective pairs of nodes;
for an individual lower-level entity of the lower-level entities:
monitoring, by the directed graph health model, values of a plurality of metrics that are collected in association with use of the individual lower-level entity; and
categorizing, by the directed graph health model, the individual lower-level entity into one of a predefined set of health categories;
determining, by the directed graph health model, a number of lower-level entities have been categorized into a particular one of the predefined set of lower-level health categories, wherein each lower-level entity in the number of lower-level entities is connected to a higher-level entity in the directed graph health model via a respective edge;
determining, by the directed graph health model, that the number of lower-level entities satisfies a threshold established to indicate that a state of the higher-level entity has changed, wherein the threshold defines a percentage of a total number of lower-level entities connected to the higher-level entity in the directed graph health model;
in response to determining that the number of lower-level entities satisfies the threshold, accessing, by the directed graph health model, a rule associated with the higher-level entity, the rule defining an action to execute for the higher-level entity when the state of the higher-level entity changes; and
executing, by the directed graph health model, the action for the higher-level entity.
16. The method of claim 15 , wherein:
the predefined set of health categories includes healthy, unhealthy, and unknown; and
the state of the higher-level entity changes to one of a healthy state, an unhealthy state, a degraded state, or an unknown state.
17. The method of claim 15 , wherein the higher-level entity, the threshold, and the rule are defined based on input from a tenant of the distributed computing environment, wherein the tenant owns the higher-level entity.