Efficient inter-chip interconnect topology for distributed parallel deep learning

1. A system comprising:

a first group of computing nodes and a second group of computing nodes, wherein the first and second groups are neighboring devices and each of the first and second groups comprising:

a set of computing nodes A-D, and

a set of intra-group interconnects, wherein the set of intra-group interconnects communicatively couple computing node A with computing nodes B and C and computing node D with computing nodes B and C; and

a set of inter-group interconnects, wherein the set of inter-group interconnects communicatively couple computing node A of the first group with computing node A of the second group, computing node B of the first group with computing node B of the second group, computing node C of the first group with computing node C of the second group, and computing node D of the first group with computing node D of the second group,

wherein the first and second groups are grouped into a plurality of sub-sections,

wherein a number of square sub-sections is maximized within the first and second groups,

wherein, for each of a plurality of incoming computing tasks to the system:

a different computing task is allocated to a different sub-section, and

a respective computing task is first allocated to a square sub-section over a non-square sub-section.

2. The system of claim 1 , wherein the set of intra-group interconnects and the set of inter-group interconnects comprise inter-chip interconnects.

3. The system of claim 2 , wherein the inter-chip interconnects of the set of intra-group interconnects or the set of inter-group interconnects are bi-directional.

4. The system of claim 3 , wherein the inter-chip interconnects are copper cables.

5. The system of claim 1 , further comprising: a third group of computing nodes, wherein the third group of computing nodes is neighboring with the second group and the third group comprising: a set of computing nodes A-D, and a set of intra-group interconnects, wherein the set of intra-group interconnects communicatively couple computing node A with computing nodes B and C and computing node D with computing nodes B and C; and a second set of inter-group interconnects, wherein the second set of inter-group interconnects communicatively couple computing node A of the third group with computing node A of the second group, computing node B of the third group with computing node B of the second group, computing node C of the third group with computing node C of the second group, and computing node D of the third group with computing node D of the second group.

6. The system of claim 5 , wherein: the intra-group interconnects and the first and second sets of inter-group interconnects form a ring connection that communicatively couples computing node A of the first group with computing node B of the first group, computing node B of the first group with computing node B of the second group, computing node B of the second group with computing node B of the third group, computing node B of the third group with computing node A of the third group, computing node A of the third group with computing node A of the second group, and computing node A of the second group with computing node A of the first group.

7. The system of claim 1 , wherein: the intra-group interconnects and the first set of inter-group interconnects form a ring connection that communicatively couples computing node A of the first group with computing node B of the first group, computing node B of the first group with computing node B of the second group, computing node B of the second group with computing node A of the second group, and computing node A of the second group with computing node A of the first group.

8. The system of claim 1 , wherein at least one sub-section comprises a ring connection that comprises a row or column of computing nodes.

9. The system of claim 1 , wherein the computing nodes are processors.

10. The system of claim 9 , wherein the computing nodes are artificial intelligence (“AT”) training processors, AI training chips, neural processing units (“NPU”), or graphic processing units (“GPU”).

11. The system of claim 1 , wherein each of the first group and the second group are boards, and each board further comprises:

a main processor; and

a hardware bridge, wherein the hardware bridge connects computing nodes A-D on the board with the main processor.

12. A method for allocating a plurality of computing tasks across a plurality of groups of computing nodes, each group of the plurality of groups comprising a set of computing nodes A-D and a set of intra-group interconnects that communicatively couple computing node A with computing nodes B and C and computing node D with computing nodes B and C, the method comprising:

acquiring the computing tasks;

determining a hardware load of the computing tasks;

allocating the computing tasks to first and second groups of the plurality of groups according to the hardware load; and

executing, using the first and second groups of the plurality of groups, the computing tasks to generate data packets that are forwarded over a set of inter-group interconnects between the first and second groups,

wherein the first and second groups are grouped into a plurality of sub-sections,

wherein a number of square sub-sections is maximized within the first and second groups,

wherein, for each of the computing tasks, allocating the computing tasks comprises:

allocating a different computing task to a different sub-section, and

first allocating a respective computing task to a square sub-section over a non-square sub-section.

13. The method of claim 12 , wherein the set of inter-group interconnects communicatively couple computing node A of the first group with computing node A of the second group, computing node B of the first group with computing node B of the second group, computing node C of the first group with computing node C of the second group, and computing node D of the first group with computing node D of the second group.

14. The method of claim 13 , wherein the set of intra-group interconnects and the set of inter-group interconnects comprise inter-chip interconnects.

15. The method of claim 12 , further comprising: creating a ring connection from intra-group interconnects and inter-group interconnects that communicatively couples computing node A of the first group with computing node B of the first group, computing node B of the first group with computing node B of the second group, computing node B of the second group with computing node A of the second group, and computing node A of the second group with computing node A of the first group.

16. The method of claim 12 , wherein the computing nodes are artificial intelligence (“AT”) training processors, AI training chips, neural processing units (“NPU”), or graphic processing units (“GPU”).

17. The method of claim 16 , wherein at least one of the computing tasks is an AI computing task.

18. The method of claim 12 , wherein the inter-chip interconnects of the set of intra-group interconnects or the set of inter-group interconnects are bi-directional.

19. The method of claim 12 , wherein each of the first group and the second group are boards, and each board further comprises:

a main processor; and

a hardware bridge, wherein the hardware bridge connects computing nodes A-D on the board with the main processor.

20. A non-transitory computer readable medium that stores a set of instructions that is executable by one or more processors of an apparatus to cause the apparatus to initiate a method for allocating a plurality of computing tasks across a plurality of groups of computing nodes, each group comprising a set of computing nodes A-D and a set of intra-group interconnects that communicatively couple computing node A with computing nodes B and C and computing node D with computing nodes B and C, the method comprising:

acquiring the computing tasks;

determining a hardware load of the computing tasks;

allocating the computing tasks to first and second groups of the plurality of groups according to the hardware load; and

executing, using the first and second groups of the plurality of groups, the computing tasks to generate data packets that are forwarded over a set of inter-group interconnects between the first and second groups,

wherein the first and second groups are grouped into a plurality of sub-sections,

wherein a number of square sub-sections is maximized within the first and second groups,

wherein, for each of the computing tasks, allocating the computing tasks comprises:

allocating a different computing task to a different sub-section, and

first allocating a respective computing task to a square sub-section over a non-square sub-section.

21. The non-transitory computer readable medium of claim 20 , wherein the set of inter-group interconnects communicatively couple computing node A of the first group with computing node A of the second group, computing node B of the first group with computing node B of the second group, computing node C of the first group with computing node C of the second group, and computing node D of the first group with computing node D of the second group.

22. The non-transitory computer readable medium of claim 21 , wherein the set of intra-group interconnects and the set of inter-group interconnects comprise inter-chip interconnects.

23. The non-transitory computer readable medium of claim 20 , the set of instructions that is executable by one or more processors of the apparatus to cause the apparatus to further perform:

creating a ring connection from intra-group interconnects and inter-group interconnects that communicatively couple computing node A of the first group with computing node B of the first group, computing node B of the first group with computing node B of the second group, computing node B of the second group with computing node A of the second group, and computing node A of the second group with computing node A of the first group.

24. The non-transitory computer readable medium of claim 20 , wherein the computing nodes are artificial intelligence (“AT”) training processors, AI training chips, neural processing units (“NPU”), or graphic processing units (“GPU”).

25. The non-transitory computer readable medium of claim 24 , wherein at least one of the computing tasks is an AI computing task.

26. The non-transitory computer readable medium of claim 20 , wherein the inter-chip interconnects of the set of intra-group interconnects or the set of inter-group interconnects are bi-directional.

27. The non-transitory computer readable medium of claim 20 , wherein each of the first group and the second group are boards, and each board further comprises:

a main processor; and

a hardware bridge, wherein the hardware bridge connects computing nodes A-D on the board with the main processor.