Implementing state change in a hierarchy of resources in an SDDC
Some embodiments provide a hierarchical data service (HDS) that manages many resource clusters that are in a resource cluster hierarchy. In some embodiments, each resource cluster has its own cluster manager, and the cluster managers are in a cluster manager hierarchy that mimics the hierarchy of the resource clusters. In some embodiments, both the resource cluster hierarchy and the cluster manager hierarchy are tree structures, e.g., a directed acyclic graph (DAG) structure that has one root node with multiple other nodes in a hierarchy, with each other node having only one parent node and one or more possible child nodes.
1. A method of hierarchically managing resources arranged in a hierarchy in at least one datacenter, the method comprising:
at a particular manager of a particular resource cluster:
receiving a set of policies to implement state change from a parent cluster manager;
after receiving the set of policies, separately receiving from the parent cluster manager a command and a set of policy-evaluating criteria, the command to implement a state change if the received set of policy-evaluating criteria satisfies a group of the received policies, wherein the set of policy-evaluating criteria comprises a predetermined threshold for available CPU cycles;
determining that the received set of policy-evaluating criteria satisfies a group of one or more received policies; and
based on said determination, implementing a state change in the particular resource cluster according to the command;
wherein the particular manager is configured to switch to a secondary parent cluster manager in response to a failure of the parent cluster manager.
2. The method of claim 1 further comprising sending a report of the state change to the parent cluster manager.
3. The method of claim 1 further comprising:
at the particular manager of the particular resource cluster:
distributing the set of policies to at least one child manager of the particular manager, the child manager managing a resource cluster that is a child resource cluster of the particular resource cluster; and
after distributing the set of policies to the child manager, separately distributing the received command and the set of policy-evaluating criteria to the child manager to use to evaluate against the distributed set of policies, in order to implement a state change on the child resource cluster according to the command based on an evaluation that the set of policy-evaluating criteria satisfy a subset of the distributed policies.
4. The method of claim 3 further comprising:
receiving a report of the state change from the child cluster manager; and
providing the report to the parent cluster manager.
5. The method of claim 1 further comprising:
at the particular manager of the particular resource cluster:
distributing the set of policies to each child manager of the particular manager, each child manager managing a different resource cluster that is a different child resource cluster of the particular resource cluster; and
after distributing the set of policies to the child manager, separately distributing the received command and the set of policy-evaluating criteria to each child manager to use to evaluate against the distributed set of policies, in order to implement a state change on the child manager's resource cluster according to the command based on an evaluation that the set of policy-evaluating criteria satisfies a subset of the distributed policies.
6. The method of claim 1 , wherein by receiving said policies, command and criteria enable the particular cluster manger to make the determination on its own as to whether the particular cluster manager needs to effectuate a state change on the particular resource cluster that it manages.
7. The method of claim 6 , wherein said enablement freeing up ancestor cluster managers of the particular cluster manager from making the determination for the particular resource cluster managed by the particular cluster manager.
8. The method of claim 1 , wherein the parent cluster manager received said policies, command and criteria from a cluster manager that is a parent of the cluster manager and a grandparent of the particular cluster manager.
9. The method of claim 1 , wherein the resource clusters comprise compute clusters and network clusters.
10. The method of claim 9 , wherein said managers are machines executing on the datacenter, said machines being one of containers, Pods, virtual machines and standalone computers.
11. A non-transitory machine readable medium storing a program for a particular manager of a particular resource cluster, the particular manage part of a hierarchical management system for managing resources arranged in a hierarchy in at least one datacenter, the program comprising sets of instructions for:
receiving a set of policies to implement state change from a parent cluster manager;
after receiving the set of policies, separately receiving from the parent cluster manager a command and a set of policy-evaluating criteria, the command to implement a state change if the received set of policy-evaluating criteria satisfies a group of the received policies, wherein the set of policy-evaluating criteria comprises a predetermined threshold for available CPU cycles;
determining that the received set of policy-evaluating criteria satisfies a group of one or more received policies; and
based on said determination, implementing a state change in the particular resource cluster according to the command;
wherein the particular manager is configured to switch to a secondary parent cluster manager in response to a failure of the parent cluster manager.
12. The non-transitory machine readable medium of claim 11 ,
wherein the program further comprising a set of instructions for sending a report of the state change to the parent cluster manager.
13. The non-transitory machine readable medium of claim 11 , wherein the program further comprising a set of instructions for:
distributing the set of policies to at least one child manager of the particular manager, the child manager managing a resource cluster that is a child resource cluster of the particular resource cluster; and
after distributing the set of policies to the child manager, separately distributing the received command and the set of policy-evaluating criteria to the child manager to use to evaluate against the distributed set of policies, in order to implement a state change on the child resource cluster according to the command based on an evaluation that the set of policy-evaluating criteria satisfy a subset of the distributed policies.
14. The non-transitory machine readable medium of claim 13 , wherein the program further comprising sets of instructions for:
receiving a report of the state change from the child cluster manager; and
providing the report to the parent cluster manager.
15. The non-transitory machine readable medium of claim 11 , wherein the program further comprising sets of instructions for:
distributing the set of policies to each child manager of the particular manager, each child manager managing a different resource cluster that is a different child resource cluster of the particular resource cluster; and
after distributing the set of policies to the child manager, separately distributing the received command and the set of policy-evaluating criteria to each child manager to use to evaluate against the distributed set of policies, in order to implement a state change on the child manager's resource cluster according to the command based on an evaluation that the set of policy-evaluating criteria satisfies a subset of the distributed policies.
16. The non-transitory machine readable medium of claim 11 , wherein by receiving said policies, command and criteria enable the particular cluster manger to make the determination on its own as to whether the particular cluster manager needs to effectuate a state change on the particular resource cluster that it manages.
17. The non-transitory machine readable medium of claim 16 , wherein said enablement freeing up ancestor cluster managers of the particular cluster manager from making the determination for the particular resource cluster managed by the particular cluster manager.
18. The non-transitory machine readable medium of claim 11 , wherein the parent cluster manager received said policies, command and criteria from a cluster manager that is a parent of the cluster manager and a grandparent of the particular cluster manager.
19. The non-transitory machine readable medium of claim 11 , wherein the resource clusters comprise compute clusters and network clusters.
20. The non-transitory machine readable medium of claim 19 , wherein said managers are machines executing on the datacenter, said machines being one of containers, Pods, virtual machines and standalone computers.