Solver for cluster management system
A processing device receives a first cluster management operation request, wherein the first cluster management operation request is for a deployment operation of a first cluster and comprises a cluster template, a specified number of nodes to deploy and a configuration. The processing device determines a plurality of constraints from at least one of the cluster template or the configuration. The processing device deterministically solves for and outputs a first cluster layout comprising the specified number of nodes, each of the specified number of nodes comprising a hardware type, an image type and one or more services, wherein the first cluster layout satisfies the plurality of constraints.
1 . A method comprising:
receiving, at a cluster management system, an assignment for deployment of a cluster including a plurality of nodes, wherein each node corresponds to one or more services running on hardware or a virtual machine, wherein the assignment comprises an initial cluster template indicating a starting state of a layout of the cluster and one or more user-specified parameters for modification of the starting state of the layout of the cluster to a modified state of the layout of the cluster;
determining, by the cluster management system, one or more node-level tasks performed for individual nodes of the plurality of nodes at a node-level granularity such that different node-level tasks are performed for different nodes of the cluster, wherein the one or more node-level tasks modify the starting state of the layout of the cluster into the modified state of the layout of the cluster, wherein the one or more node-level tasks are determined based on the initial cluster template and the one or more user-specified parameters, wherein different node-level tasks are associated with different user-specified parameters;
determining, by the cluster management system, an ordered execution plan for the one or more node-level tasks; and
instructing, by the cluster management system, one or more provisioners to execute the one or more node-level tasks according to the ordered execution plan.
2 . The method of claim 1 , wherein the user-specified parameters include configuration data indicating one or more constraints to the initial cluster template.
3 . The method of claim 2 , wherein the one or more constraints include at least one of a service, an image type, or a hardware type.
4 . The method of claim 1 , further comprising receiving, by the cluster management system, a target cluster layout, wherein determining one or more node-level tasks is based further on the target cluster layout.
5 . The method of claim 1 , further comprising determining, by the cluster management system, a dependency between two or more node-level tasks, wherein determining the ordered execution plan is based on the determined dependency such that a first node-level task that is dependent on a second node-level task is started after the second node-level task is started.
6 . The method of claim 1 , wherein the ordered execution plan comprises two or more node-level tasks to be performed in parallel.
7 . The method of claim 1 , further comprising:
recording a log of performed node-level tasks; and
in response to execution of a node-level task of the ordered execution plan failing, creating a new execution plan for rolling back the cluster management operation from a then-current cluster layout to a starting cluster layout, wherein the new execution plan comprises one or more new node-level tasks; and
instructing the one or more provisioners to execute one or more new node-level tasks according to the new execution plan.
8 . The method of claim 7 , wherein the one or more new node-level tasks are not an inverse of the performed node-level tasks in the recorded log.
9 . The method of claim 1 , further comprising:
receiving, by a first provisioner of the one or more provisioners, a first node-level task, wherein the first node-level task is associated with a first user-specified parameter indicating a provider for the cluster management operation; and
executing, by the first provisioner, the first node-level task through a provider-specific application-program interface (API) of the provider indicated in the first user-specified parameter.
10 . The method of claim 9 , further comprising:
issuing, by the first provisioner, a request to execute the first node-level task, wherein the first node-level task creates a new node;
receiving, by the first provisioner, a provider identification of the new node in response to the request to execute the first node-level task;
reporting, by the first provisioner, the provider identification of the new node to the cluster management system;
after reporting the provider identification of the new node to the cluster management system, receiving from the cluster management system, by the first provisioner, an instruction to execute a second node-level task, wherein the second node-level task is associated with the new node, and wherein the instruction to execute the second node-level task includes the provider identification of the new node.
11 . A cluster management system comprising:
one or more computing processors; and
a memory containing instructions, the instructions when executed configured to perform the steps of:
receiving an assignment for deployment of a cluster including a plurality of nodes, wherein each node corresponds to one or more services running on hardware or a virtual machine, wherein the assignment comprises an initial cluster template indicating a starting state of a layout of the cluster and one or more user-specified parameters for modification of the starting state of the layout of the cluster to a modified state of the layout of the cluster; and
determining one or more node-level tasks performed for individual nodes of the plurality of nodes at a node-level granularity such that different node-level tasks are performed for different nodes of the cluster, wherein the one or more node-level tasks modify the starting state of the layout of the cluster into the modified state of the layout of the cluster, wherein the one or more node-level tasks are determined based on the initial cluster template and the one or more user-specified parameters, wherein different node-level tasks are associated with different user-specified parameters;
determining an ordered execution plan for the one or more node-level tasks; and
instructing one or more provisioners to execute the one or more node-level tasks according to the ordered execution plan.
12 . The cluster management system of claim 11 , wherein the user-specified parameters include configuration data indicating one or more constraints to the initial cluster template.
13 . The cluster management system of claim 12 , wherein the one or more constraints include at least one of a service, an image type, or a hardware type.
14 . The cluster management system of claim 11 , further comprising the instructions configured to perform the step of receiving a target cluster layout, wherein determining one or more node-level tasks is based further on the target cluster layout.
15 . The cluster management system of claim 11 , further comprising the instructions configured to perform the step of determining a dependency between two or more node-level tasks, wherein determining the ordered execution plan is based on the determined dependency such that a first node-level task that is dependent on a second node-level task is started after the second node-level task is started.
16 . The cluster management system of claim 11 , wherein the ordered execution plan comprises two or more node-level tasks to be performed in parallel.
17 . The cluster management system of claim 11 , further comprising the instructions configured to perform the steps of:
recording a log of performed node-level tasks; and
in response to execution of a node-level task of the ordered execution plan failing, creating a new execution plan for rolling back the cluster management operation from a then-current cluster layout to a starting cluster layout, wherein the new execution plan comprises one or more new node-level tasks; and
instructing the one or more provisioners to execute one or more new node-level tasks according to the new execution plan.
18 . The cluster management system of claim 17 , wherein the one or more new node-level tasks are not an inverse of the performed node-level tasks in the recorded log.
19 . A system comprising:
the cluster management system of claim 11 ; and
the one or more provisioners, wherein a first provisioner of the one or more provisioners is configured to:
receive a first node-level task, wherein the first node-level task is associated with a first user-specified parameter indicating a provider for the cluster management operation; and
execute the first node-level task through a provider-specific application-program interface (API) of the provider indicated in the first user-specified parameter.
20 . The system of claim 19 , further comprising:
issue a request to execute the first node-level task, wherein the first node-level task creates a new node;
receive a provider identification of the new node in response to the request to execute the first node-level task;
report the provider identification of the new node to the cluster management system;
after reporting the provider identification of the new node to the cluster management system, receive from the cluster management system an instruction to execute a second node-level task, wherein the second node-level task is associated with the new node, and wherein the instruction to execute the second node-level task includes the provider identification of the new node.
21 . The method of claim 1 , wherein the one or more node-level tasks include one or a combination of: adding one or more nodes to the cluster, adding one or more services to one or more existing nodes of the cluster at the node-level granularity, deleting one or more nodes from the cluster, and deleting one or more services to one or more existing nodes of the cluster at the node-level granularity.