Supercluster network of graphical processing units (GPUS)
Described herein is a network fabric including a plurality of graphical processing unit (GPU) clusters. The plurality of GPU clusters includes at least a first GPU cluster operating at a first speed and a second GPU cluster operating at a second speed that is different than the first speed. The network fabric includes a plurality of blocks, wherein each block includes: (a) one or more racks that host a GPU cluster, and (b) a plurality of switches arranged in a hierarchical structure that communicatively couple the block to other blocks included in the network fabric. Responsive to receiving a request to execute a workload, allocating one or more GPUs from the plurality of GPU clusters to execute the workload.
1. A method comprising:
providing a network fabric including: (i) a plurality of graphical processing unit (GPU) clusters, the plurality of GPU clusters including at least a first GPU cluster operating at a first speed and a second GPU cluster operating at a second speed that is different than the first speed, and (ii) a plurality of blocks, wherein each block includes: (a) one or more racks that host a GPU cluster, and (b) a plurality of switches arranged in a hierarchical structure that communicatively couple the block to other blocks included in the network fabric; and
responsive to receiving a request to execute a workload, allocating one or more GPUs from the plurality of GPU clusters to execute the workload.
2. The method of claim 1 , wherein the network fabric includes:
a first block hosting the first GPU cluster operating at the first speed, the first GPU cluster including a first number of GPUs, and
a second block hosting the second GPU cluster operating at the second speed, the second GPU cluster including a second number of GPUs, and wherein each GPU included in the first GPU cluster operates at the first speed, and each GPU included in the second GPU cluster operates at the second speed.
3. The method of claim 2 , wherein each of the first block and the second block of the plurality of blocks includes:
a first tier of switches of the hierarchical structure, wherein each switch included in the first tier of switches has a first dimension, and
a second tier of switches of the hierarchical structure, wherein each switch included in the second tier of switches has a second dimension that is different than the first dimension, and wherein the first tier of switches are communicatively coupled at one end to the one or more racks and at another end to the second tier of switches.
4. The method of claim 3 , wherein the first dimension of a first switch included in the first tier of switches in the first block corresponds to a first number of downstream ports and a second number of upstream ports, wherein each port included in the first number of downstream ports and each port included in the second number of upstream ports operates at the first speed.
5. The method of claim 3 , wherein the second dimension of a second switch included in the second tier of switches corresponds to a third number of downstream ports and a fourth number of upstream ports, wherein each port included in the third number of downstream ports and each port included in the fourth number of upstream ports operates at the second speed.
6. The method of claim 5 , wherein each port of the third number of downstream ports of the second switch included in the first block is split into multiple sub-ports, each sub-port being communicatively coupled to a unique switch in the first tier of switches of the first block.
7. The method of claim 3 , wherein the first dimension of a third switch included in the first tier of switches in the second block corresponds to a fifth number of downstream ports and a sixth number of upstream ports, wherein each port included in the fifth number of downstream ports and each port included in the sixth number of upstream ports operates at the second speed.
8. The method of claim 2 , wherein the first GPU cluster included in the first block operates at the first speed of 100 G, and the second GPU cluster included in the second block operates at the second speed of 400 G.
9. The method of claim 3 , wherein the network fabric further includes a plurality of groups of third tier of switches, wherein a first group of third tier of switches communicatively couples the first block to the second block.
10. The method of claim 9 , wherein each switch included in the first group of third tier of switches includes ports that operate at the second speed and are communicatively coupled to the second tier of switches included in the first block and the second block.
11. One or more computer readable non-transitory media storing computer-executable instructions that, when executed by one or more processors, cause:
providing a network fabric including: (i) a plurality of graphical processing unit (GPU) clusters, the plurality of GPU clusters including at least a first GPU cluster operating at a first speed and a second GPU cluster operating at a second speed that is different than the first speed, and (ii) a plurality of blocks, wherein each block includes: (a) one or more racks that host a GPU cluster, and (b) a plurality of switches arranged in a hierarchical structure that communicatively couple the block to other blocks included in the network fabric; and
responsive to receiving a request to execute a workload, allocating one or more GPUs from the plurality of GPU clusters to execute the workload.
12. The one or more computer readable non-transitory media storing computer-executable instructions of claim 11 , wherein the network fabric includes:
a first block hosting the first GPU cluster operating at the first speed, the first GPU cluster including a first number of GPUs, and
a second block hosting the second GPU cluster operating at the second speed, the second GPU cluster including a second number of GPUs, and wherein each GPU included in the first GPU cluster operates at the first speed, and each GPU included in the second GPU cluster operates at the second speed.
13. The one or more computer readable non-transitory media storing computer-executable instructions of claim 12 , wherein each of the first block and the second block of the plurality of blocks includes:
a first tier of switches of the hierarchical structure, wherein each switch included in the first tier of switches has a first dimension, and
a second tier of switches of the hierarchical structure, wherein each switch included in the second tier of switches has a second dimension that is different than the first dimension, and wherein the first tier of switches are communicatively coupled at one end to the one or more racks and at another end to the second tier of switches.
14. The one or more computer readable non-transitory media storing computer-executable instructions of claim 13 , wherein the first dimension of a first switch included in the first tier of switches in the first block corresponds to a first number of downstream ports and a second number of upstream ports, wherein each port included in the first number of downstream ports and each port included in the second number of upstream ports operates at the first speed.
15. The one or more computer readable non-transitory media storing computer-executable instructions of claim 13 , wherein the second dimension of a second switch included in the second tier of switches corresponds to a third number of downstream ports and a fourth number of upstream ports, wherein each port included in the third number of downstream ports and each port included in the fourth number of upstream ports operates at the second speed.
16. The one or more computer readable non-transitory media storing computer-executable instructions of claim 15 , wherein each port of the third number of downstream ports of the second switch included in the first block is split into multiple sub-ports, each sub-port being communicatively coupled to a unique switch in the first tier of switches of the first block.
17. The one or more computer readable non-transitory media storing computer-executable instructions of claim 13 , wherein the first dimension of a third switch included in the first tier of switches in the second block corresponds to a fifth number of downstream ports and a sixth number of upstream ports, wherein each port included in the fifth number of downstream ports and each port included in the sixth number of upstream ports operates at the second speed.
18. The one or more computer readable non-transitory media storing computer-executable instructions of claim 12 , wherein the first GPU cluster included in the first block operates at the first speed of 100 G, and the second GPU cluster included in the second block operates at the second speed of 400 G.
19. The one or more computer readable non-transitory media storing computer-executable instructions of claim 13 , wherein the network fabric further includes a plurality of groups of third tier of switches, wherein a first group of third tier of switches communicatively couples the first block to the second block.
20. A computing device comprising:
one or more processors; and
a memory including instructions that, when executed with the one or more processors, cause the computing device to, at least:
provide a network fabric including: (i) a plurality of graphical processing unit (GPU) clusters, the plurality of GPU clusters including at least a first GPU cluster operating at a first speed and a second GPU cluster operating at a second speed that is different than the first speed, and (ii) a plurality of blocks, wherein each block includes: (a) one or more racks that host a GPU cluster, and (b) a plurality of switches arranged in a hierarchical structure that communicatively couple the block to other blocks included in the network fabric; and
responsive to receiving a request to execute a workload, allocate one or more GPUs from the plurality of GPU clusters to execute the workload.