Scheduling jobs on graphical processing units

1. A method comprising:

receiving, in a computing system including one or more graphics processing units (GPUs), the one or more GPUs providing concurrent processing of a plurality of jobs by a plurality of virtual GPUs (vGPUs), a request to schedule a new job to be executed by the computing system;

in response to receiving the request:

determining a set of job allocations for allocating a set of jobs to the plurality of vGPUs that minimizes a cost function comprising a GPU operational cost component and a job migration cost component, wherein the set of jobs comprises the new job and one or more existing jobs that have previously been allocated to the plurality of vGPUs;

allocating the new job to one or more vGPUs of the plurality of vGPUs according to the determined set of allocations; and

updating allocations of the existing jobs to one or more vGPUs of the plurality of vGPUs according to the determined set of allocations; and

processing the new job and the existing jobs by the one or more GPUs in the computing system,

wherein determining the set of allocations that minimizes cost function comprises minimizing the cost function subject to constraints of: a number of jobs allocated to a GPU cannot be more than a capacity of the GPU, the new job can be scheduled on only one GPU, and migration of a job is performed when a new allocation of an existing job is different than a current allocation of an existing job.

2. The method of claim 1 , wherein the GPU operational cost component is indicative of an expected cost of operating the one or more GPUs according to the set of allocations and the job migration cost component is indicative of expected costs of job migrations resulting from the set of allocations.

3. The method of claim 1 , wherein updating allocations of the existing jobs according to the determined set of allocations comprises migrating an existing job from one GPU of the one or more GPUs to another GPU of the one or more GPUs.

4. The method of claim 1 , further comprising:

in response to one of the existing jobs completing, determining a second set of allocations for a second set of jobs that minimizes the cost function, the second set of jobs comprising the existing jobs excluding the completed one of the existing jobs.

5. The method of claim 1 , comprising receiving weights for the GPU operational cost component and job migration cost component.

6. The method of claim 1 , wherein determining the set of allocations that minimizes the cost function comprises:

receiving, by a GPU scheduler, inputs comprising: an identification of the set of jobs, previous allocation decisions of the existing jobs, weights for migration costs of new and existing jobs, weights for the GPU operational cost component and the job migration cost component, a number of vGPUs required for each job, and a number of GPUs in the computing system; and

minimizing the cost function based on the inputs.

7. The method of claim 1 , wherein determining the set of allocations that minimizes the cost function comprises determining, by a GPU scheduler, decisions for allocating new and existing jobs, a number of job migrations and migration cost, decisions on migrating existing jobs, and decisions on powering off/on GPUs.

8. A non-transitory machine-readable storage medium having stored thereon executable instructions that, when executed by a processing resource, cause the processing resource to:

receive, in a computing system including one or more graphics processing units (GPUs), the one or more GPUs providing concurrent processing of a plurality of jobs by a plurality of virtual GPUs (vGPUs), a request to schedule a new job to be executed by the computing system;

in response to receiving the request:

determine a set of job allocations for allocating a set of jobs to the plurality of vGPUs that minimizes a cost function comprising a GPU operational cost component and a migration cost component;

wherein the set of jobs comprises the new job and one or more existing jobs that have previously been allocated to the plurality of vGPUs;

allocate the new job to one or more vGPUs of the plurality of vGPUs according to the determined set of allocations; and

update allocations of the existing jobs to one or more vGPUs of the plurality of vGPUs according to the determined set of allocations; and

process the new job and the existing jobs by the one or more GPUs in the computing system,

wherein determining the set of allocations that minimizes cost function comprises minimizing the cost function subject to constraints of: a number of jobs allocated to a GPU cannot be more than a capacity of the GPU, the new job can be scheduled on only one GPU, and migration of a job is performed when a new allocation of an existing job is different than a current allocation of an existing job.

9. The non-transitory machine-readable storage medium of claim 8 , wherein the GPU operational cost component is indicative of an expected cost of operating the one or more GPUs according to the set of allocations and the job migration cost component is indicative of expected costs of job migrations resulting from the set of allocations.

10. The non-transitory machine-readable storage medium of claim 8 , wherein instructions to cause the processing resource to update allocations of the existing jobs according to the determined set of allocations comprise instructions to cause the processing resource to migrate an existing job from one GPU of the one or more GPUs to another GPU of the one or more GPUs.

11. The non-transitory machine-readable storage medium of claim 8 , further comprising instructions to cause the processing resource to, in response to one of the existing jobs completing, determining a second set of job allocations for a second set of jobs that minimizes the cost function, the second set of jobs comprising the existing jobs excluding the completed one of the existing jobs.

12. The non-transitory machine-readable storage medium of claim 8 , comprising instructions that, when executed, cause the processing resource to receive weights for the operational cost and migration cost.

13. A computing system comprising:

one or more graphics processing units (GPUs), the one or more GPUs providing concurrent processing of a plurality of jobs by a plurality of virtual GPUs (vGPUs); and

a GPU scheduler to receive a request to schedule a new job to be executed by the computing system and in response to receiving the request:

determine a set of job allocations for allocating a set of jobs to the plurality of vGPUs that minimizes a cost function comprising a GPU operational cost component and a job migration cost component,

wherein the set of jobs comprises the new job and one or more existing jobs that have previously been allocated to the plurality of vGPUs; and

allocate the new job to one or more vGPUs of the plurality of vGPUs according to the determined set of allocations,

update allocations of the existing jobs to one or more vGPUs of the plurality of vGPUs according to the determined set of allocations;

wherein the one or more GPUs to process the new job and the existing jobs, and

wherein determining the set of allocations that minimizes cost function comprises minimizing the cost function subject to constraints of: a number of jobs allocated to a GPU cannot be more than a capacity of the GPU, the new job can be scheduled on only one GPU, and migration of a job is performed when a new allocation of an existing job is different than a current allocation of an existing job.

14. The computing system of claim 13 , wherein the GPU operational cost component is indicative of an expected cost of operating the one or more GPUs according to the set of allocations and the job migration cost component is indicative of expected costs of job migrations resulting from the set of allocations.

15. The computing system of claim 13 , wherein the GPU scheduler updating allocations of the existing jobs comprises the GPU scheduler migrating an existing job from one GPU of the one or more GPUs to another GPU of the one or more GPUs.

16. The computing system of claim 13 , wherein the GPU scheduler is to, in response to one of the existing jobs completing, determine a second set of job allocations for a second set of jobs that minimizes the cost function, the second set of jobs comprising the existing jobs excluding the completed one of the existing jobs.

17. The computing system of claim 13 , wherein the GPU scheduler is to receive weights for the GPU operational cost component and job migration cost component.

18. The computing system of claim 13 , wherein the GPU scheduler determining the set of allocations that minimizes cost function comprises:

receiving, by the GPU scheduler, inputs comprising: an identification of the set of jobs, previous allocation decisions of the existing jobs, weights for migration costs of new and existing jobs, weights for the GPU operational cost component and weights for the job migration cost component, a number of vGPUs required for each job, and a number of GPUs in the computing system; and

minimizing the cost function based on the inputs.

19. The computing system of claim 13 , wherein the GPU scheduler determining the set of allocations that minimizes the cost function comprises determining, by the GPU scheduler, decisions for allocating new and existing jobs, a number of job migrations and migration cost, decisions on migrating existing jobs, and decisions on powering off/on GPUs.