Dynamic kernel slicing for VGPU sharing in serverless computing systems

1. A system for dynamic kernel slicing in a serverless computing service, comprising:

at least one computing device;

program instructions stored in memory and executable in the at least one computing device that, when executed by the at least one computing device, direct the at least one computing device to:

receive a request for execution of program code in the serverless computing service in which a plurality of virtual graphics processing units (vGPUs) are used in the execution of the program code on an underlying physical graphics processing unit (GPU);

in response to receipt of the request for execution of the program code, determine a slice size to partition a compute kernel of the program code into a plurality of sub-kernels for concurrent execution by the vGPUs, the slice size being determined for individual ones of the sub-kernels using an optimization function in accordance with a scheduling policy;

determine an execution schedule for executing the individual ones of the sub-kernels on the vGPUs in accordance with the scheduling policy; and

execute the sub-kernels on the vGPUs as partitioned in accordance with the execution schedule.

2. The system of claim 1 , wherein:

the request for execution of the program code is associated with an execution deadline; and

the individual ones of the sub-kernels are scheduled in accordance with the scheduling policy based at least in part on the execution deadline.

3. The system of claim 2 , wherein the execution schedule and the slice size are determined as a function of at least two of:

the execution deadline, a number of blocks in a task of the compute kernel, an arrival time of the task, a start time of a workload switch on the GPU, an end time of the workload switch on the GPU, a load at the time of the workload switch on the GPU, a total number of load switches on the GPU, a set of possible slice sizes for the task, a total number of workloads, an expected execution time of one of the sub-kernels, an elapsed time, and a total number of tasks to be scheduled.

4. The system of claim 3 , wherein:

the scheduling policy maximizes a number of tasks completed with the execution deadline; and

the execution schedule and the slice size are determined using the optimization function in accordance with the scheduling policy.

5. The system of claim 4 , wherein the optimization function comprises an integer non-linear program (INLP).

6. The system of claim 5 , wherein the scheduling policy is further determined using a round-robin routine or a priority-based routine.

7. The system of claim 1 , wherein the at least one computing device is further directed to assign the individual ones of the sub-kernels to a corresponding one of a plurality of containers of at least one virtual machine.

8. A method for dynamic kernel slicing in a serverless computing service, comprising:

receiving a request for execution of program code in the serverless computing service in which a plurality of virtual graphics processing units (vGPUs) are used in the execution of the program code on an underlying physical graphics processing unit (GPU);

in response to receipt of the request for execution of the program code, determining a slice size to partition a compute kernel of the program code into a plurality of sub-kernels for concurrent execution by the vGPUs, the slice size being determined for individual ones of the sub-kernels using an optimization function in accordance with a scheduling policy;

determining an execution schedule for executing the individual ones of the sub-kernels on the vGPUs in accordance with the scheduling policy; and

executing the sub-kernels on the vGPUs as partitioned in accordance with the execution schedule.

9. The method of claim 8 , wherein:

the request for execution of the program code is associated with an execution deadline; and

the individual ones of the sub-kernels are scheduled in accordance with the scheduling policy based at least in part on the execution deadline.

10. The method of claim 9 , wherein the execution schedule and the slice size are determined as a function of at least two of:

the execution deadline, a number of blocks in a task of the compute kernel, an arrival time of the task, a start time of a workload switch on the GPU, an end time of the workload switch on the GPU, a load at the time of the workload switch on the GPU, a total number of load switches on the GPU, a set of possible slice sizes for the task, a total number of workloads, an expected execution time of one of the sub-kernels, an elapsed time, and a total number of tasks to be scheduled.

11. The method of claim 10 , wherein:

the scheduling policy maximizes a number of tasks completed with the execution deadline; and

the execution schedule and the slice size are determined using the optimization function in accordance with the scheduling policy.

12. The method of claim 11 , wherein the optimization function comprises an integer non-linear program (INLP).

13. The method of claim 12 , wherein the scheduling policy is further determined using a round-robin routine or a priority-based routine.

14. The method of claim 8 , further comprising assigning the individual ones of the sub-kernels to a corresponding one of a plurality of containers of at least one virtual machine.

15. A non-transitory computer-readable medium comprising program instructions for dynamic kernel slicing in a serverless computing service that, when executed by at least one computing device, direct the at least one computing device to:

receive a request for execution of program code in the serverless computing service in which a plurality of virtual graphics processing units (vGPUs) are used in the execution of the program code on an underlying physical graphics processing unit (GPU);

in response to receipt of the request for execution of the program code, determine a slice size to partition a compute kernel of the program code into a plurality of sub-kernels for concurrent execution by the vGPUs, the slice size being determined for individual ones of the sub-kernels using an optimization function in accordance with a scheduling policy;

determine an execution schedule for executing the individual ones of the sub-kernels on the vGPUs in accordance with the scheduling policy; and

execute the sub-kernels on the vGPUs as partitioned in accordance with the execution schedule.

16. The non-transitory computer-readable medium of claim 15 , wherein:

the request for execution of the program code is associated with an execution deadline; and

the individual ones of the sub-kernels are scheduled in accordance with the scheduling policy based at least in part on the execution deadline.

17. The non-transitory computer-readable medium of claim 16 , wherein the execution schedule and the slice size are determined as a function of at least two of:

the execution deadline, a number of blocks in a task of the compute kernel, an arrival time of the task, a start time of a workload switch on the GPU, an end time of the workload switch on the GPU, a load at the time of the workload switch on the GPU, a total number of load switches on the GPU, a set of possible slice sizes for the task, a total number of workloads, an expected execution time of one of the sub-kernels, an elapsed time, and a total number of tasks to be scheduled.

18. The non-transitory computer-readable medium of claim 17 , wherein:

the scheduling policy maximizes a number of tasks completed with the execution deadline; and

the execution schedule and the slice size are determined using the optimization function in accordance with the scheduling policy.

19. The non-transitory computer-readable medium of claim 18 , wherein the optimization function comprises an integer non-linear program (INLP).

20. The non-transitory computer-readable medium of claim 15 , wherein the at least one computing device is further directed to assign the individual ones of the sub-kernels to a corresponding one of a plurality of containers of at least one virtual machine.