OPTIMAL MULTI-INSTANCE GPU (MIG) AWARE PLACEMENT OF CLIENTS
In one set of embodiments, a computer system can receive a plurality of requests for placing a plurality of clients on a plurality of graphics processing units (GPUs), where each request includes a profile specifying a number of GPU compute slices and a number of GPU memory slices requested by a corresponding client. The computer system can further formulate an integer linear programming (ILP) problem based on the requests and a maximum number of GPU compute and memory slices supported by each GPU. The computer system can then generate a solution for the ILP problem and place the plurality of clients on the plurality of GPUs in accordance with the solution.
1 . A method comprising:
receiving, by a computer system, a plurality of requests for placing a plurality of clients on a plurality of graphics processing units (GPUs), each request including a profile specifying a number of GPU compute slices and a number of GPU memory slices requested by a corresponding client;
determining, by the computer system, an optimal placement of the plurality of clients on the plurality of GPUs, the determining comprising:
formulating an integer linear programming (ILP) problem that includes:
a set of constants corresponding to a maximum number of GPU compute slices and a maximum number of GPU memory slices supported by each GPU;
a set of coefficients corresponding to the number of GPU compute slices and the number of GPU memory slices requested by each client;
a set of decision variables indicating whether a given client is placed on a given GPU;
a set of constraints that is based on the set of constants, the set of coefficients, and the set of decision variables; and
an objective function that is based on the set of decision variables and computes a total number of GPUs used; and
generating a solution for the ILP problem using an ILP solver, the solution including values for the decision variables that minimize the objective function while satisfying the set of constraints; and
placing, by the computer system, the plurality of clients on the plurality of GPUs in accordance with the solution.
2 . The method of claim 1 wherein the computer system is a virtual infrastructure management (VIM) server, wherein the plurality of clients are virtual machines (VMs), and wherein the plurality of GPUs reside in a host cluster managed by the VIM server.
3 . The method of claim 1 wherein the profile is a multi-instance GPU (MIG) profile and wherein the plurality of GPUs support MIG.
4 . The method of claim 1 wherein the set of constraints includes:
a first constraint that ensures a number of GPU memory slices allocated to each client is greater than or equal to the number of GPU memory slices requested by the client; and
a second constraint that ensures a number of GPU compute slices allocated to each client is greater than or equal to the number of GPU compute slices requested by the client.
5 . The method of claim 1 wherein the set of constraints includes:
a first constraint that ensures a total number of GPU memory slices allocated on each GPU is less than or equal to the maximum number of GPU memory slices supported by the GPU; and
a second constraint that ensures a total number of GPU compute slices allocated on each GPU is less than or equal to the maximum number of GPU compute slices supported by the GPU.
6 . The method of claim 1 wherein the set of constraints includes a constraint that ensures all GPU compute and memory slices allocated to each client are allocated on a single GPU.
7 . The method of claim 1 wherein the set of constraints includes a constraint that ensures each client is placed on at most one GPU.
8 . A non-transitory computer readable storage medium having stored thereon program code executable by a computer system, the program code causing the computer system to execute a method comprising:
receiving a plurality of requests for placing a plurality of clients on a plurality of graphics processing units (GPUs), each request including a profile specifying a number of GPU compute slices and a number of GPU memory slices requested by a corresponding client;
determining an optimal placement of the plurality of clients on the plurality of GPUs, the determining comprising:
formulating an integer linear programming (ILP) problem that includes:
a set of constants corresponding to a maximum number of GPU compute slices and a maximum number of GPU memory slices supported by each GPU;
a set of coefficients corresponding to the number of GPU compute slices and the number of GPU memory slices requested by each client;
a set of decision variables indicating whether a given client is placed on a given GPU;
a set of constraints that is based on the set of constants, the set of coefficients, and the set of decision variables; and
an objective function that is based on the set of decision variables and computes a total number of GPUs used; and
generating a solution for the ILP problem using an ILP solver, the solution including values for the decision variables that minimize the objective function while satisfying the set of constraints; and
placing the plurality of clients on the plurality of GPUs in accordance with the solution.
9 . The non-transitory computer readable storage medium of claim 8 wherein the computer system is a virtual infrastructure management (VIM) server, wherein the plurality of clients are virtual machines (VMs), and wherein the plurality of GPUs reside in a host cluster managed by the VIM server.
10 . The non-transitory computer readable storage medium of claim 8 wherein the profile is a multi-instance GPU (MIG) profile and wherein the plurality of GPUs support MIG.
11 . The non-transitory computer readable storage medium of claim 8 wherein the set of constraints includes:
a first constraint that ensures a number of GPU memory slices allocated to each client is greater than or equal to the number of GPU memory slices requested by the client; and
a second constraint that ensures a number of GPU compute slices allocated to each client is greater than or equal to the number of GPU compute slices requested by the client.
12 . The non-transitory computer readable storage medium of claim 8 wherein the set of constraints includes:
a first constraint that ensures a total number of GPU memory slices allocated on each GPU is less than or equal to the maximum number of GPU memory slices supported by the GPU; and
a second constraint that ensures a total number of GPU compute slices allocated on each GPU is less than or equal to the maximum number of GPU compute slices supported by the GPU.
13 . The non-transitory computer readable storage medium of claim 8 wherein the set of constraints includes a constraint that ensures all GPU compute and memory slices allocated to each client are allocated on a single GPU.
14 . The non-transitory computer readable storage medium of claim 8 wherein the set of constraints includes a constraint that ensures each client is placed on at most one GPU.
15 . A computer system comprising:
a processor; and
a non-transitory computer readable medium having stored thereon program code that, when executed, causes the processor to:
receive a plurality of requests for placing a plurality of clients on a plurality of graphics processing units (GPUs), each request including a profile specifying a number of GPU compute slices and a number of GPU memory slices requested by a corresponding client;
determine an optimal placement of the plurality of clients on the plurality of GPUS, the determining comprising:
formulating an integer linear programming (ILP) problem that includes:
a set of constants corresponding to a maximum number of GPU compute slices and a maximum number of GPU memory slices supported by each GPU;
a set of coefficients corresponding to the number of GPU compute slices and the number of GPU memory slices requested by each client;
a set of decision variables indicating whether a given client is placed on a given GPU;
a set of constraints that is based on the set of constants, the set of coefficients, and the set of decision variables; and
an objective function that is based on the set of decision variables and computes a total number of GPUs used; and
generating a solution for the ILP problem using an ILP solver, the solution including values for the decision variables that minimize the objective function while satisfying the set of constraints; and
place the plurality of clients on the plurality of GPUs in accordance with the solution.
16 . The computer system of claim 15 wherein the computer system is a virtual infrastructure management (VIM) server, wherein the plurality of clients are virtual machines (VMs), and wherein the plurality of GPUs reside in a host cluster managed by the VIM server.
17 . The computer system of claim 15 wherein the profile is a multi-instance GPU (MIG) profile and wherein the plurality of GPUs support MIG.
18 . The computer system of claim 15 wherein the set of constraints includes:
a first constraint that ensures a number of GPU memory slices allocated to each client is greater than or equal to the number of GPU memory slices requested by the client; and
a second constraint that ensures a number of GPU compute slices allocated to each client is greater than or equal to the number of GPU compute slices requested by the client.
19 . The computer system of claim 15 wherein the set of constraints includes:
a first constraint that ensures a total number of GPU memory slices allocated on each GPU is less than or equal to the maximum number of GPU memory slices supported by the GPU; and
a second constraint that ensures a total number of GPU compute slices allocated on each GPU is less than or equal to the maximum number of GPU compute slices supported by the GPU.
20 . The computer system of claim 15 wherein the set of constraints includes a constraint that ensures all GPU compute and memory slices allocated to each client are allocated on a single GPU.
21 . The computer system of claim 15 wherein the set of constraints includes a constraint that ensures each client is placed on at most one GPU.