Policy controller for distributed virtualization infrastructure element monitoring
This disclosure describes techniques for monitoring, scheduling, and performance management for virtualization infrastructures within networks. In one example, a computing system includes a plurality of different cloud-based compute clusters (e.g., different cloud projects), each comprising a set of compute nodes. Policy agents execute on the compute nodes to monitor performance and usage metrics relating to resources of the compute nodes. Policy controllers within each cluster deploy policies to the policy agents and evaluate performance and usage metrics from the policy agents by application of one or more rulesets for infrastructure elements of the compute cluster. Each of the policy controllers outputs data to a multi-cluster dashboard software system indicative of a current health status for the infrastructure elements based on the evaluation of the performance and usage metrics for the cluster. The multi-cluster dashboard software system data outputs, as single user interface screen, the current health status for each of the cloud-based compute clusters.
1. A method comprising:
enabling, by a controller, a plurality of policy agents to monitor usage metrics for each of a first plurality of compute nodes within a first compute cluster and to monitor usage metrics for each of a second plurality of compute nodes within a second compute cluster;
receiving, by the controller and from the plurality of policy agents, information about the usage metrics for the first plurality of compute nodes and information about the usage metrics for the second plurality of compute nodes;
evaluating, by the controller and based on a first ruleset applied to the information about the usage metrics for the first plurality of compute nodes, infrastructure elements for the first compute cluster;
evaluating, by the controller and based on a second ruleset applied to the information about the usage metrics for the second plurality of compute nodes, infrastructure elements for the second compute cluster;
generating, by the controller, health data including a first health status for the infrastructure elements for the first cluster and a second health status for the infrastructure elements for the second cluster; and
outputting, by the controller, a user interface for presentation as a multi-cluster dashboard, wherein the user interface presents information about the first health status and the second health status, and wherein outputting the user interface includes enabling presentation of a single user interface display having a plurality of graphical regions arranged within the single user interface display, wherein the single user interface display presents information about the first health status within one of the graphical regions and information about the second health status within a different one of the graphical regions.
2. The method of claim 1 , wherein outputting the user interface includes:
enabling presentation of a plurality of graphical tiles within each of the plurality of graphical regions; and
wherein each of the graphical tiles corresponds to a different type of infrastructure element and includes an integer count of infrastructure elements being monitored, and
wherein each of the graphical tiles has an appearance indicating whether the particular type of infrastructure element has been determined to be unhealthy.
3. The method of claim 1 , wherein outputting the user interface includes:
outputting a first single-cluster user interface for presentation as a single-cluster dashboard for the first cluster, wherein the first single-cluster user interface presents health status information about the first compute cluster but not health status information about the second compute cluster; and
outputting a second single-cluster user interface for presentation as a single-cluster dashboard for the second cluster, wherein the second single-cluster user interface presents health status information about the second compute cluster but not health status information about the first compute cluster.
4. The method of claim 3 , further comprising:
enabling, by the controller, switching between the multi-cluster dashboard, the single-cluster dashboard for the first cluster, and the single-cluster dashboard for the second cluster without requiring re-authentication of a user.
5. The method of claim 3 , further comprising:
maintaining, by the controller, a hash table storing authentication tokens for each of the compute clusters;
switching, by the controller and responsive to input, between the multi-cluster dashboard, the single-cluster dashboard for the first cluster, and the single-cluster dashboard for the second cluster without requiring reauthentication of a user by utilizing the authentication tokens from the hash table.
6. The method of claim 1 ,
wherein the first ruleset includes a first set of alarms; and
wherein the second ruleset includes a second set of alarms.
7. The method of claim 6 ,
wherein the first ruleset includes a first set of assigned weights for the first set of alarms; and
wherein the second ruleset includes a second set of assigned weights for the second set of alarms.
8. The method of claim 7 , wherein generating the health data includes:
generating health data by applying the first set of assigned weights to the first set of alarms, and by applying the second assigned weights to the second set of alarms.
9. The method of claim 1 , wherein the first health status is a current first health status, wherein the second health status is a current second health status, and wherein generating health data includes:
generating first difference information specifying changes to a prior first health status of infrastructure elements for the first cluster; and
generating second difference information specifying changes a prior second health status of infrastructure elements for the second cluster.
10. The method of claim 9 ,
wherein the first difference information includes specifying a count of a number of infrastructure elements for the first cluster having a changed health status; and
wherein the second difference information includes specifying a count of a number of infrastructure elements for the second cluster having a changed health status.
11. A controller comprising a storage system and processing circuitry, wherein the processing circuitry has access to the storage system and is configured to:
enable a plurality of policy agents to monitor usage metrics for each of a first plurality of compute nodes within a first compute cluster and to monitor usage metrics for each of a second plurality of compute nodes within a second compute cluster;
receive, from the plurality of policy agents, information about the usage metrics for the first plurality of compute nodes and information about the usage metrics for the second plurality of compute nodes;
evaluate, based on a first ruleset applied to the information about the usage metrics for the first plurality of compute nodes, infrastructure elements for the first compute cluster;
evaluate, based on a second ruleset applied to the information about the usage metrics for the second plurality of compute nodes, infrastructure elements for the second compute cluster;
generate health data including a first health status for the infrastructure elements for the first cluster and a second health status for the infrastructure elements for the second cluster; and
output a user interface for presentation as a multi-cluster dashboard, wherein the user interface presents information about the first health status and the second health status, and wherein to output the user interface, the processing circuitry is further configured to enable presentation of a single user interface display having a plurality of graphical regions arranged within the single user interface display, wherein the single user interface display presents information about the first health status within one of the graphical regions and information about the second health status within a different one of the graphical regions.
12. The controller of claim 11 , wherein to output the user interface, the processing circuitry is further configured to:
enable presentation of a plurality of graphical tiles within each of the plurality of graphical regions; and
wherein each of the graphical tiles corresponds to a different type of infrastructure element and includes an integer count of infrastructure elements being monitored, and
wherein each of the graphical tiles has an appearance indicating whether the particular type of infrastructure element has been determined to be unhealthy.
13. The controller of claim 11 , wherein to output the user interface, the processing circuitry is further configured to:
output a first single-cluster user interface for presentation as a single-cluster dashboard for the first cluster, wherein the first single-cluster user interface presents health status information about the first compute cluster but not health status information about the second compute cluster; and
output a second single-cluster user interface for presentation as a single-cluster dashboard for the second cluster, wherein the second single-cluster user interface presents health status information about the second compute cluster but not health status information about the first compute cluster.
14. The controller of claim 13 , wherein the processing circuitry is further configured to:
enable, switching between the multi-cluster dashboard, the single-cluster dashboard for the first cluster, and the single-cluster dashboard for the second cluster without requiring re-authentication of a user.
15. The controller of claim 13 , wherein the processing circuitry is further configured to:
maintain a hash table storing authentication tokens for each of the compute clusters;
switch, responsive to input, between the multi-cluster dashboard, the single-cluster dashboard for the first cluster, and the single-cluster dashboard for the second cluster without requiring reauthentication of a user by utilizing the authentication tokens from the hash table.
16. The controller of claim 11 ,
wherein the first ruleset includes a first set of alarms; and
wherein the second ruleset includes a second set of alarms.
17. The controller of claim 16 ,
wherein the first ruleset includes a first set of assigned weights for the first set of alarms; and
wherein the second ruleset includes a second set of assigned weights for the second set of alarms.
18. A non-transitory computer-readable medium comprising instructions that, when executed, configure processing circuitry of a computing system to:
enable a plurality of policy agents to monitor usage metrics for each of a first plurality of compute nodes within a first compute cluster and to monitor usage metrics for each of a second plurality of compute nodes within a second compute cluster;
receive, from the plurality of policy agents, information about the usage metrics for the first plurality of compute nodes and information about the usage metrics for the second plurality of compute nodes;
evaluate, based on a first ruleset applied to the information about the usage metrics for the first plurality of compute nodes, infrastructure elements for the first compute cluster;
evaluate, based on a second ruleset applied to the information about the usage metrics for the second plurality of compute nodes, infrastructure elements for the second compute cluster;
generate health data including a first health status for the infrastructure elements for the first cluster and a second health status for the infrastructure elements for the second cluster; and
output a user interface for presentation as a multi-cluster dashboard, wherein the user interface presents information about the first health status and the second health status, and wherein to output the user interface, the processing circuitry is further configured to enable presentation of a single user interface display having a plurality of graphical regions arranged within the single user interface display, wherein the single user interface display presents information about the first health status within one of the graphical regions and information about the second health status within a different one of the graphical regions.