Repairing high-availability virtual machine placements in a computing cluster
Placement scenario optimization mechanisms for automatic placement of computing entities onto nodes of a running multi-node computing cluster. A set of failure mode parameters define a high-availability requirement of the multi-node computing cluster. In advance of a failure event, and responsive to a determination that a then-current computing entity placement does not satisfy the high-availability requirement, the cluster is analyzed and a plurality of feasible placement scenarios are generated. Optimization criteria are applied to the feasible placement scenarios such that a best choice from among the feasible placement scenarios is identified and applied to the virtual machine placements over the cluster. A change monitoring and detection facility continually observes the multi-node computing cluster to detect a change of a failure mode parameter or to detect a change to the configuration of the virtual machines. Certain of such changes cause feasible placement scenarios to be generated, evaluated, selected, and applied.
1. A non-transitory computer readable medium having stored thereon a sequence of instructions which, when stored in memory and executed by a processor cause the processor to perform acts comprising:
establishing a first set of virtualization system failure mode parameters that define a high availability requirement of a virtualization system that forms a multi-node computing cluster;
establishing a first virtual machine (VM) placement configuration wherein the first VM placement configuration corresponds to the first set of virtualization system failure mode parameters, and wherein the first VM placement configuration corresponds to a first high availability compliance state that meets or exceeds the high availability requirement;
operating the virtualization system, at least until the virtualization system undergoes a transition from operating according to the first set of virtualization system failure mode parameters to operating according to a second set of virtualization system failure mode parameters; and
before a failure occurs, and responsive to the transition in the virtualization system, automatically establishing a second VM placement configuration wherein the second VM placement configuration corresponds to the second set of virtualization system failure mode parameters, and wherein the second VM placement configuration, when operating according to the second set of virtualization system failure mode parameters, corresponds to a second high availability compliance state of the virtualization system, and wherein the second high availability compliance state of the virtualization system is different from the first high availability compliance state of the virtualization system.
2. The non-transitory computer readable medium of claim 1 , wherein the first VM placement configuration, when operating according to the second set of virtualization system failure mode parameters, does not meet the high availability requirement.
3. The non-transitory computer readable medium of claim 1 , wherein a change to a failure mode parameter of the multi-node computing cluster causes the virtualization system to transition from operating according to the first set of virtualization system failure mode parameters to operating according to the second set of virtualization system failure mode parameters.
4. The non-transitory computer readable medium of claim 3 , wherein the change to the failure mode parameter is a change from a one-node fault tolerance mode to a two-node fault tolerance mode.
5. The non-transitory computer readable medium of claim 3 , wherein the second VM placement configuration that is different from the first VM placement configuration corresponds to a change to at least one of, a number of virtual machines running on the multi-node computing cluster, a number of nodes that constitute the multi-node computing cluster, a number of cores of a node, an amount of allocable memory of a node, a number of virtual machines of a node, an amount of headroom of a node, or an amount of network bandwidth demand of a node.
6. The non-transitory computer readable medium of claim 1 , wherein, in response to occurrence of a maintenance operation over the multi-node computing cluster, the virtualization system transitions from operating according to the first set of virtualization system failure mode parameters to operating according to the second set of virtualization system failure mode parameters.
7. The non-transitory computer readable medium of claim 6 , wherein the maintenance operation comprises at least one of, a removal of a node of the multi-node computing cluster, or an addition of a node to the multi-node computing cluster.
8. The non-transitory computer readable medium of claim 1 , wherein, in response to occurrence of a callback from one or more configuration change application programming interfaces (APIs), the virtualization system transitions from operating according to the first set of virtualization system failure mode parameters to operating according to the second set of virtualization system failure mode parameters.
9. A method comprising:
establishing a first set of virtualization system failure mode parameters that define a high availability requirement of a virtualization system that forms a multi-node computing cluster;
establishing a first virtual machine (VM) placement configuration wherein the first VM placement configuration corresponds to the first set of virtualization system failure mode parameters, and wherein the first VM placement configuration corresponds to a first high availability compliance state that meets or exceeds the high availability requirement;
operating the virtualization system, at least until the virtualization system undergoes a transition from operating according to the first set of virtualization system failure mode parameters to operating according to a second set of virtualization system failure mode parameters; and
before a failure occurs, and responsive to the transition in the virtualization system, automatically establishing a second VM placement configuration wherein the second VM placement configuration corresponds to the second set of virtualization system failure mode parameters, and wherein the second VM placement configuration, when operating according to the second set of virtualization system failure mode parameters, corresponds to a second high availability compliance state of the virtualization system, and wherein the second high availability compliance state of the virtualization system is different from the first high availability compliance state of the virtualization system.
10. The method of claim 9 , wherein the first VM placement configuration, when operating according to the second set of virtualization system failure mode parameters, does not meet the high availability requirement.
11. The method of claim 9 , wherein a change to a failure mode parameter of the multi-node computing cluster causes the virtualization system to transition from operating according to the first set of virtualization system failure mode parameters to operating according to the second set of virtualization system failure mode parameters.
12. The method of claim 11 , wherein the change to the failure mode parameter is a change from a one-node fault tolerance mode to a two-node fault tolerance mode.
13. The method of claim 11 , wherein the second VM placement configuration that is different from the first VM placement configuration corresponds to a change to at least one of, a number of virtual machines running on the multi-node computing cluster, a number of nodes that constitute the multi-node computing cluster, a number of cores of a node, an amount of allocable memory of a node, a number of virtual machines of a node, an amount of headroom of a node, or an amount of network bandwidth demand of a node.
14. The method of claim 9 , wherein, in response to occurrence of a maintenance operation over the multi-node computing cluster, the virtualization system transitions from operating according to the first set of virtualization system failure mode parameters to operating according to the second set of virtualization system failure mode parameters.
15. The method of claim 14 , wherein the maintenance operation comprises at least one of, a removal of a node of the multi-node computing cluster, or an addition of a node to the multi-node computing cluster.
16. The method of claim 9 , wherein, in response to occurrence of a callback from one or more configuration change application programming interfaces (APIs), the virtualization system transitions from operating according to the first set of virtualization system failure mode parameters to operating according to the second set of virtualization system failure mode parameters.
17. A system comprising:
a storage medium having stored thereon a sequence of instructions; and
a processor that executes the sequence of instructions to cause the processor to perform acts comprising,
establishing a first set of virtualization system failure mode parameters that define a high availability requirement of a virtualization system that forms a multi-node computing cluster;
establishing a first virtual machine (VM) placement configuration wherein the first VM placement configuration corresponds to the first set of virtualization system failure mode parameters, and wherein the first VM placement configuration corresponds to a first high availability compliance state that meets or exceeds the high availability requirement;
operating the virtualization system, at least until the virtualization system undergoes a transition from operating according to the first set of virtualization system failure mode parameters to operating according to a second set of virtualization system failure mode parameters; and
before a failure occurs, and responsive to the transition in the virtualization system, automatically establishing a second VM placement configuration wherein the second VM placement configuration corresponds to the second set of virtualization system failure mode parameters, and wherein the second VM placement configuration, when operating according to the second set of virtualization system failure mode parameters, corresponds to a second high availability compliance state of the virtualization system, and wherein the second high availability compliance state of the virtualization system is different from the first high availability compliance state of the virtualization system.
18. The system of claim 17 , wherein the first VM placement configuration, when operating according to the second set of virtualization system failure mode parameters, does not meet the high availability requirement.
19. The system of claim 17 , wherein a change to a failure mode parameter of the multi-node computing cluster causes the virtualization system to transition from operating according to the first set of virtualization system failure mode parameters to operating according to the second set of virtualization system failure mode parameters.
20. The system of claim 19 , wherein the change to the failure mode parameter is a change from a one-node fault tolerance mode to a two-node fault tolerance mode.
21. The system of claim 19 , wherein the second VM placement configuration that is different from the first VM placement configuration corresponds to a change to at least one of, a number of virtual machines running on the multi-node computing cluster, a number of nodes that constitute the multi-node computing cluster, a number of cores of a node, an amount of allocable memory of a node, a number of virtual machines of a node, an amount of headroom of a node, or an amount of network bandwidth demand of a node.
22. The system of claim 17 , wherein, in response to occurrence of a maintenance operation over the multi-node computing cluster, the virtualization system transitions from operating according to the first set of virtualization system failure mode parameters to operating according to the second set of virtualization system failure mode parameters.
23. The system of claim 22 , wherein the maintenance operation comprises at least one of, a removal of a node of the multi-node computing cluster, or an addition of a node to the multi-node computing cluster.
24. The system of claim 17 , wherein, in response to occurrence of a callback from one or more configuration change application programming interfaces (APIs), the virtualization system transitions from operating according to the first set of virtualization system failure mode parameters to operating according to the second set of virtualization system failure mode parameters.