Topology aware grouping and provisioning of GPU resources in GPU-as-a-Service platform

1. A method, comprising:

receiving, by a graphics processing unit (GPU) server node, a service request from a client system for GPU processing services provided by the GPU server node, wherein the GPU server node comprises a plurality of GPU devices;

determining a hardware interconnect topology of the GPU server node, the hardware interconnect topology comprising information regarding determined interconnect paths between the GPU devices of the GPU server node and between the GPU devices and hardware components of the GPU server node;

accessing a performance metrics data structure which comprises performance metrics associated with a plurality of different interconnect path types that can be used to connect to GPU devices in a GPU server node topology, wherein the performance metrics comprise predefined priority scores that accord different priorities to the plurality of different interconnect path types;

dynamically forming a group of GPU devices for handling the service request received from the client system based at least in part on the determined hardware interconnect topology of the GPU server node and the performance metrics, wherein the group of GPU devices is dynamically formed at least in part by selecting one or more GPU devices of the GPU server node which are determined to be interconnected with higher priority interconnect paths as compared to other GPU devices of the GPU server node which are determined to be interconnected with lower priority interconnect paths; and

provisioning the dynamically formed group of GPU devices to the client system to handle the service request.

2. The method of claim 1 , further comprising accessing one or more quality of service policies associated with the client system, wherein the group of GPU devices is dynamically formed based at least in part on the one or more quality of service policies associated with the client system.

3. The method of claim 1 , wherein the performance metrics comprise a predefined priority score that is accorded to a type of interconnect path that is used to connect a GPU device and a network adapter which is utilized to connect to the GPU server node.

4. The method of claim 1 , wherein the performance metrics comprise a predefined priority score that is accorded to a type of interconnect path that is used to connect two GPU devices of the GPU server node.

5. The method of claim 1 , wherein the performance metrics comprise a predefined priority score that is accorded to a type of interconnect path which includes a processor socket-level link, wherein the priority accorded to the type of interconnect path which includes a processor socket-level link is lower than a priority accorded to a type of interconnect path which does not include a processor socket-level link.

6. The method of claim 1 , further comprising:

generating a system topology data structure comprising information regarding the determined hardware interconnect topology of the GPU server node; and

populating the system topology data structure with priority information for the determined interconnect paths included in the system topology data structure based on the predefined priority scores of the performance metrics data structure.

7. The method of claim 1 , wherein dynamically forming the group of GPU devices of the GPU server node comprises selecting one or more GPU devices that are already included as part of one or more other dynamically formed groups of GPU devices provisioned to other client systems.

8. An article of manufacture comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code is executable by a processor to implement a process comprising:

receiving, by a graphics processing unit (GPU) server node, a service request from a client system for GPU processing services provided by the GPU server node, wherein the GPU server node comprises a plurality of GPU devices;

determining a hardware interconnect topology of the GPU server node, the hardware interconnect topology comprising information regarding determined interconnect paths between the GPU devices of the GPU server node and between the GPU devices and hardware components of the GPU server node;

accessing a performance metrics data structure which comprises performance metrics associated with a plurality of different interconnect path types that can be used to connect to GPU devices in a GPU server node topology, wherein the performance metrics comprise predefined priority scores that accord different priorities to the plurality of different interconnect path types;

dynamically forming a group of GPU devices for handling the service request received from the client system based at least in part on the determined hardware interconnect topology of the GPU server node and the performance metrics, wherein the group of GPU devices is dynamically formed at least in part by selecting one or more GPU devices of the GPU server node which are determined to be interconnected with higher priority interconnect paths as compared to other GPU devices of the GPU server node which are determined to be interconnected with lower priority interconnect paths; and

provisioning the dynamically formed group of GPU devices to the client system to handle the service request.

9. The article of manufacture of claim 8 , further comprising program code that is executable by the processor to perform a method comprising accessing one or more quality of service policies associated with the client system, wherein the group of GPU devices is dynamically formed based at least in part on the one or more quality of service policies associated with the client system.

10. The article of manufacture of claim 8 , wherein the performance metrics comprise a predefined priority score that is accorded to a type of interconnect path that is used to connect a GPU device and a network adapter which is utilized to connect to the GPU server node.

11. The article of manufacture of claim 8 , wherein the performance metrics comprise a predefined priority score that is accorded to a type of interconnect path that is used to connect two GPU devices of the GPU server node.

12. The article of manufacture of claim 8 , wherein the performance metrics comprise a predefined priority score that is accorded to a type of interconnect path which includes a processor socket-level link, wherein the priority accorded to the type of interconnect path which includes a processor socket-level link is lower than a priority accorded to a type of interconnect path which does not include a processor socket-level link.

13. The article of manufacture of claim 8 , further comprising program code that is executable by the processor to perform a method comprising:

generating a system topology data structure comprising information regarding the determined hardware interconnect topology of the GPU server node; and

populating the system topology data structure with priority information for the determined interconnect paths included in the system topology data structure based on the predefined priority scores of the performance metrics data structure.

14. The article of manufacture of claim 8 , wherein dynamically forming the group of GPU devices of the GPU server node comprises selecting one or more GPU devices that are already included as part of one or more other dynamically formed groups of GPU devices provisioned to other client systems.

15. A graphics processing unit (GPU) server node, comprising:

a plurality of GPU devices;

a memory to store program instructions; and

a processor to execute the stored program instructions to cause the GPU server node to perform a process which comprises:

receiving a service request from a client system for GPU processing services provided by the GPU server node;

determining a hardware interconnect topology of the GPU server node, the hardware interconnect topology comprising information regarding determined interconnect paths between the GPU devices of the GPU server node and between the GPU devices and hardware components of the GPU server node;

accessing a performance metrics data structure which comprises performance metrics associated with a plurality of different interconnect path types that can be used to connect to GPU devices in a GPU server node topology, wherein the performance metrics comprise predefined priority scores that accord different priorities to the plurality of different interconnect path types;

dynamically forming a group of GPU devices for handling the service request received from the client system based at least in part on the determined hardware interconnect topology of the GPU server node and the performance metrics, wherein the group of GPU devices is dynamically formed at least in part by selecting one or more GPU devices of the GPU server node which are determined to be interconnected with higher priority interconnect paths as compared to other GPU devices of the GPU server node which are determined to be interconnected with lower priority interconnect paths; and

provisioning the dynamically formed group of GPU devices to the client system to handle the service request.

16. The GPU server node of claim 15 , therein the process performed by the GPU server node further comprises accessing one or more quality of service policies associated with the client system, wherein the group of GPU devices is dynamically formed based at least in part on the one or more quality of service policies associated with the client system.

17. The GPU server node of claim 15 , wherein the performance metrics comprise a predefined priority score that is accorded to a type of interconnect path that is used to connect a GPU device and a network adapter which is utilized to connect to the GPU server node.

18. The GPU server node of claim 15 , wherein the performance metrics comprise a predefined priority score that is accorded to a type of interconnect path that is used to connect two GPU devices of the GPU server node.

19. The GPU server node of claim 15 , wherein the performance metrics comprise a predefined priority score that is accorded to a type of interconnect path which includes a processor socket-level link, wherein the priority accorded to the type of interconnect path which includes a processor socket-level link is lower than a priority accorded to a type of interconnect path which does not include a processor socket-level link.

20. The GPU server node of claim 15 , therein the process performed by the GPU server node further comprises:

generating a system topology data structure comprising information regarding the determined hardware interconnect topology of the GPU server node; and

populating the system topology data structure with priority information for the determined interconnect paths included in the system topology data structure based on the predefined priority scores of the performance metrics data structure.