Load balancing virtual computing instances associated with virtual graphics processing units
Various embodiments set forth one or more non-transitory computer-readable media storing program instructions that, when executed by one or more processors, cause the one or more processors to perform steps of determining at least one physical resource of a node in a cluster of nodes is under contention by virtual computing instances or a virtual computing instance cannot be placed on the cluster of nodes, determining a placement for one or more virtual computing instances on the cluster of nodes, each virtual computing instance having a virtual resource associated with a profile that is compatible with a profile associated with a physical resource of a node on which the virtual computing instance is placed, and generating and executing a plan to achieve the placement, the plan including at least one of migrating at least one virtual computing instance or reconfiguring a profile associated with at least one physical resource.
1 . One or more non-transitory computer-readable media storing program instructions that, when executed by one or more processors, cause the one or more processors to perform steps of:
determining a new placement for one or more virtual computing instances on a cluster of nodes is required;
determining the new placement for the one or more virtual computing instances on the cluster of nodes, each virtual computing instance having a virtual resource associated with a physical graphics processing unit (pGPU) of a node on which the virtual computing instance is placed, wherein a profile associated with the pGPU is selected from a predefined set of profiles that the pGPU is reconfigurable with to support different profiles associated with virtual resources included in virtual computing instances; and
generating and executing a plan to achieve the new placement, wherein generating and executing the plan comprises:
determining or selecting the plan based on at least one of: the plan including a minimum number of virtual computing instance migrations, or the plan including a minimum amount of migrated memory for the virtual computing instance migrations;
migrating at least one virtual computing instance away from at least one pGPU to enable a reconfiguration of the pGPU specified in the plan;
reconfiguring a profile for the at least one pGPU after migrating the at least one virtual computing instance; and
executing the one or more virtual computing instances using the at least one pGPU to complete the new placement.
2 . The one or more non-transitory computer-readable media of claim 1 , wherein a first virtual resource of a first virtual computing instance is a virtual graphics processing unit, and a first physical resource of a first node on which the first virtual computing instance is placed is a pGPU.
3 . The one or more non-transitory computer-readable media of claim 1 , wherein, when at least one pGPU of a first node in the cluster of nodes is under contention, the new placement comprises placing a first virtual computing instance that is executing on the first node onto a second node.
4 . The one or more non-transitory computer-readable media of claim 1 , wherein the at least one pGPU includes a first pGPU of a first node, and wherein migrating the at least one virtual computing instance comprises consolidating a plurality of virtual computing instances on one or more nodes to free the first pGPU.
5 . The one or more non-transitory computer-readable media of claim 1 , wherein the new placement is determined based on one or more placement constraints that include at least one of a constraint that different virtual resources of a virtual computing instance are assigned to different pGPUs, a constraint that a number of virtual computing instances having virtual resources associated with respective profiles that are assigned to each pGPU does not exceed a capacity of the pGPU, a constraint that each pGPU is associated with at most one profile, or a constraint that a virtual computing instance having a virtual resource that is assigned to a pGPU of a node cannot be migrated to a same node.
6 . The one or more non-transitory computer-readable media of claim 1 , wherein the new placement is determined based on at least one of a cost function or a heuristic that first places virtual computing instances having at least one of a greatest number of a first type of virtual resource or a greatest size of one or more virtual resources.
7 . The one or more non-transitory computer-readable media of claim 1 , wherein the plan is generated to satisfy at least one of a rule that virtual resources assigned to each pGPU do not exceed a capacity of pGPU or a rule that, prior to assigning a virtual resource to a pGPU associated with a profile that must be reconfigured to support the virtual resource, all virtual computing instances having virtual resources assigned to the pGPU are migrated to other nodes.
8 . The one or more non-transitory computer-readable media of claim 1 , wherein the plan is generated to include a minimum number of migrations of virtual computing instances.
9 . The one or more non-transitory computer-readable media of claim 1 , wherein the one or more virtual computing instances comprise at least one virtual machine or container.
10 . The one or more non-transitory computer-readable media of claim 1 , wherein the cluster of nodes is included in a public cloud computing system, a private cloud computing system, or a hybrid cloud computing system.
11 . The one or more non-transitory computer-readable media of claim 1 , wherein determining the new placement for the one or more virtual computing instances on the cluster of nodes is required comprises determining at least one pGPU of a node in the cluster of nodes is under contention by a set of virtual computing instances executing on the node.
12 . The one or more non-transitory computer-readable media of claim 1 , wherein determining the new placement for the one or more virtual computing instances on the cluster of nodes is required comprises determining a new virtual computing instance cannot be placed on any node in the cluster of nodes.
13 . The one or more non-transitory computer-readable media of claim 1 , wherein the new placement is determined based on a cost, and wherein the steps further include computing the cost based on a number of the at least one virtual computing instance that is migrated.
14 . The one or more non-transitory computer-readable media of claim 1 , wherein the plan further includes an ordering for migrating one or more virtual computing instances that causes one or more capacity constraints associated with one or more pGPUs to be satisfied continuously.
15 . A computer-implemented method for scheduling one or more virtual computing instances, the method comprising:
determining a new placement for one or more virtual computing instances on a cluster of nodes is required;
determining the new placement for the one or more virtual computing instances on the cluster of nodes, each virtual computing instance having a virtual resource associated with a physical graphics processing unit (pGPU) of a node on which the virtual computing instance is placed, wherein a profile associated with the pGPU is selected from a predefined set of profiles that the pGPU is reconfigurable with to support different profiles associated with virtual resources included in virtual computing instances; and
generating and executing a plan to achieve the new placement, wherein generating and executing the plan comprises:
determining or selecting the plan based on at least one of: the plan including a minimum number of virtual computing instance migrations, or the plan including a minimum amount of migrated memory for the virtual computing instance migrations;
migrating at least one virtual computing instance away from at least one pGPU to enable a reconfiguration of the pGPU specified in the plan;
reconfiguring a profile for the at least one pGPU after migrating the at least one virtual computing instance; and
executing the one or more virtual computing instances using the at least one pGPU to complete the new placement.
16 . The computer-implemented method of claim 15 , wherein a first virtual resource of a first virtual computing instance is a virtual graphics processing unit, and a first physical resource of a first node on which the first virtual computing instance is placed is a pGPU.
17 . The computer-implemented method of claim 15 , wherein, when at least one pGPU of a first node in the cluster of nodes is under contention, the new placement comprises placing a first virtual computing instance that is executing on the first node onto a second node.
18 . The computer-implemented method of claim 15 , The one or more non-transitory computer-readable media of claim 1 , wherein the at least one pGPU includes a first pGPU of a first node, and wherein migrating the at least one virtual computing instance comprises consolidating a plurality of virtual computing instances on one or more nodes to free the first pGPU.
19 . The computer-implemented method of claim 15 , wherein the new placement is determined based on one or more placement constraints that include at least one of a constraint that different virtual resources of a virtual computing instance are assigned to different pGPUs, a constraint that a number of virtual computing instances having virtual resources associated with respective profiles that are assigned to each pGPU does not exceed a capacity of the pGPU, a constraint that each pGPU is associated with at most one profile, or a constraint that a virtual computing instance having a virtual resource that is assigned to a pGPU of a node cannot be migrated to a same node.
20 . The computer-implemented method of claim 15 , wherein the new placement is determined based on at least one of a cost function or a heuristic that first places virtual computing instances having at least one of a greatest number of a first type of virtual resource or a greatest size of one or more virtual resources.
21 . The computer-implemented method of claim 15 , wherein the plan is generated to satisfy at least one of a rule that virtual resources assigned to each pGPU do not exceed a capacity of the pGPU or a rule that, prior to assigning a virtual resource to a pGPU associated with a profile that must be reconfigured to support the virtual resource, all virtual computing instances having virtual resources assigned to the pGPU are migrated to other nodes.
22 . The computer-implemented method of claim 15 , wherein the plan is generated to include a minimum number of migrations of virtual computing instances.
23 . The computer-implemented method of claim 15 , wherein determining the new placement for the one or more virtual computing instances on the cluster of nodes is required comprises determining at least one pGPU of a node in the cluster of nodes is under contention by a set of virtual computing instances executing on the node.
24 . The computer-implemented method of claim 15 , wherein determining the new placement for the one or more virtual computing instances on the cluster of nodes is required comprises determining a new virtual computing instance cannot be placed on any node in the cluster of nodes.
25 . The computer-implemented method of claim 15 , wherein the new placement is determined based on a cost, and wherein the method further comprises computing the cost based on a number of the at least one virtual computing instance that is migrated.
26 . The computer-implemented method of claim 15 , wherein the plan further includes an ordering for migrating one or more virtual computing instances that causes one or more capacity constraints associated with one or more pGPUs to be satisfied continuously.
27 . A system, comprising:
one or more memories that include instructions; and
one or more processors that are coupled to the one or more memories and, when executing the instructions:
determine a new placement for one or more virtual computing instances on a cluster of nodes is required,
determine the new placement for the one or more virtual computing instances on the cluster of nodes, each virtual computing instance having a virtual resource associated with a physical graphics processing unit (pGPU) of a node on which the virtual computing instance is placed, wherein a profile associated with the pGPU is selected from a predefined set of profiles that the pGPU is reconfigurable with to support different profiles associated with virtual resources included in virtual computing instances, and
generating and executing a plan to achieve the new placement, wherein generating and executing the plan comprises:
determining or selecting the plan based on at least one of: the plan including a minimum number of virtual computing instance migrations, or the plan including a minimum amount of migrated memory for the virtual computing instance migrations;
migrating at least one virtual computing instance away from at least one pGPU to enable a reconfiguration of the pGPU specified in the plan;
reconfiguring a profile for the at least one pGPU after migrating the at least one virtual computing instance; and
executing the one or more virtual computing instances using the at least one pGPU to complete the new placement.
28 . The system of claim 27 , wherein a first virtual resource of a first virtual computing instance is a virtual graphics processing unit, and a first physical resource of a first node on which the first virtual computing instance is placed is a pGPU.
29 . The system of claim 27 , wherein, when at least one pGPU of a first node in the cluster of nodes is under contention, the new placement comprises placing a first virtual computing instance that is executing on the first node onto a second node.
30 . The system of claim 27 , wherein the at least one pGPU includes a first pGPU of a first node, and wherein migrating the at least one virtual computing instance comprises consolidating a plurality of virtual computing instances on one or more nodes to free the first pGPU.
31 . The system of claim 27 , wherein the new placement is determined based on one or more placement constraints that include at least one of a a constraint that different virtual resources of a virtual computing instance are assigned to different pGPUs, a constraint that a number of virtual computing instances having virtual resources associated with respective profiles that are assigned to each pGPU does not exceed a capacity of the pGPU, a constraint that each pGPU is associated with at most one profile, or a constraint that a virtual computing instance having a virtual resource that is assigned to a pGPU of a node cannot be migrated to a same node.
32 . The system of claim 27 , wherein the new placement is determined based on at least one of a cost function or a heuristic that first places virtual computing instances having at least one of a greatest number of a first type of virtual resource or a greatest size of one or more virtual resources.
33 . The system of claim 27 , wherein the plan is generated to satisfy at least one of a rule that virtual resources assigned to each pGPU do not exceed a capacity of the pGPU or a rule that, prior to assigning a virtual resource to a pGPU associated with a profile that must be reconfigured to support the virtual resource, all virtual computing instances having virtual resources assigned to the pGPU are migrated to other nodes.
34 . The system of claim 27 , wherein the plan is generated to include a minimum number of migrations of virtual computing instances.
35 . The system of claim 27 , wherein determining the new placement for the one or more virtual computing instances on the cluster of nodes is required comprises determining at least one pGPU of a node in the cluster of nodes is under contention by a set of virtual computing instances executing on the node.
36 . The system of claim 27 , wherein determining the new placement for the one or more virtual computing instances on the cluster of nodes is required comprises determining a new virtual computing instance cannot be placed on any node in the cluster of nodes.
37 . The system of claim 27 , wherein the new placement is determined based on a cost, and wherein method further comprises computing the cost based on a number of the at least one virtual computing instance that is migrated.
38 . The system of claim 27 , wherein the plan further includes an ordering for migrating one or more virtual computing instances that causes one or more capacity constraints associated with one or more pGPUs to be satisfied continuously.