Processing unit virtualization with scalable over-provisioning in an information processing system
Techniques are disclosed for processing unit virtualization with scalable over-provisioning in an information processing system. For example, the method accesses a data structure that maps a correspondence between a plurality of virtualized processing units and a plurality of abstracted processing units, wherein the plurality of abstracted processing units are configured to decouple an allocation decision from the plurality of virtualized processing units, and further wherein at least one of the virtualized processing units is mapped to multiple ones of the abstracted processing units. The method allocates one or more virtualized processing units to execute a given application by allocating one or more abstracted processing units identified from the data structure. The method also enables migration of one or more virtualized processing units across the system. Examples of processing units with which scalable over-provisioning functionality can be applied include, but are not limited to, accelerators such as GPUs.
1. An apparatus comprising:
a processing platform comprising at least one processor coupled to at least one memory, the processing platform, when executing program code, is configured to:
obtain a request from a client for allocation of one or more virtualized processing units to execute at least a portion of a given application, wherein the one or more virtualized processing units are part of a plurality of virtualized processing units implemented on one or more servers;
access a data structure that maps a correspondence between the plurality of virtualized processing units and a plurality of abstracted processing units, wherein the plurality of abstracted processing units are configured to decouple an allocation decision from the plurality of virtualized processing units, and further wherein at least one of the virtualized processing units is mapped to multiple ones of the abstracted processing units; and
allocate one or more virtualized processing units to execute the portion of the given application by allocating one or more abstracted processing units identified from the data structure.
2. The apparatus of claim 1 , wherein an allocated one of the one or more virtualized processing units is allocated but idle.
3. The apparatus of claim 1 , wherein an allocated one of the one or more virtualized processing units is not allocated.
4. The apparatus of claim 1 , wherein an allocated one of the one or more virtualized processing units corresponds to an allocated one of the abstracted processing units whose quality of service can be degraded.
5. The apparatus of claim 1 , wherein the processing platform, when executing program code, is further configured to update the data structure based on the allocation.
6. The apparatus of claim 1 , wherein the processing platform, when executing program code, is further configured to notify the client about the allocation.
7. The apparatus of claim 1 , wherein the processing platform, when executing program code, is further configured to, in response to completion of execution of the at least a portion of a given application, migrate one or more abstracted processing units across the one or more servers according to a load balancing policy.
8. The apparatus of claim 1 , wherein subsets of the plurality of virtualized processing units are each partitioned to a given physical processing unit of a plurality of physical processing units deployed in the one or more servers.
9. The apparatus of claim 8 , wherein the plurality of physical processing units comprises a plurality of accelerators.
10. The apparatus of claim 9 , wherein the plurality of accelerators comprises a plurality of graphics processing units.
11. The apparatus of claim 1 , wherein the given application comprises an artificial intelligence/machine learning application.
12. A method comprising:
obtaining a request from a client for allocation of one or more virtualized processing units to execute at least a portion of a given application, wherein the one or more virtualized processing units are part of a plurality of virtualized processing units implemented on one or more servers;
accessing a data structure that maps a correspondence between the plurality of virtualized processing units and a plurality of abstracted processing units, wherein the plurality of abstracted processing units are configured to decouple an allocation decision from the plurality of virtualized processing units, and further wherein at least one of the virtualized processing units is mapped to multiple ones of the abstracted processing units; and
allocating one or more virtualized processing units to execute the portion of the given application by allocating one or more abstracted processing units identified from the data structure;
wherein the obtaining, accessing, and allocating steps are performed by a processing platform comprising at least one processor coupled to at least one memory executing program code.
13. The method of claim 12 , wherein an allocated one of the one or more virtualized processing units is allocated but idle.
14. The method of claim 12 , wherein an allocated one of the one or more virtualized processing units is not allocated.
15. The method of claim 12 , wherein an allocated one of the one or more virtualized processing units corresponds to an allocated one of the abstracted processing units whose quality of service can be degraded.
16. The method of claim 12 , further comprising updating the data structure based on the allocation.
17. The method of claim 12 , further comprising notifying the client about the allocation.
18. The method of claim 12 , further comprising, in response to completion of execution of the at least a portion of a given application, migrating one or more abstracted processing units across the one or more servers according to a load balancing policy.
19. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device cause the at least one processing device to:
obtain a request from a client for allocation of one or more virtualized processing units to execute at least a portion of a given application, wherein the one or more virtualized processing units are part of a plurality of virtualized processing units implemented on one or more servers;
access a data structure that maps a correspondence between the plurality of virtualized processing units and a plurality of abstracted processing units, wherein the plurality of abstracted processing units are configured to decouple an allocation decision from the plurality of virtualized processing units, and further wherein at least one of the virtualized processing units is mapped to multiple ones of the abstracted processing units; and
allocate one or more virtualized processing units to execute the portion of the given application by allocating one or more abstracted processing units identified from the data structure.
20. The computer program product of claim 19 , further comprising, in response to completion of execution of the at least a portion of a given application, migrating one or more abstracted processing units across the one or more servers according to a load balancing policy.