Granted Patent B2

US 12,289,213 · App. 18/721,620 · Granted Apr 29, 2025

Inventors: Fang Cao (Shandong, CN); Zhenhua Guo (Shandong, CN); Li Wang (Shandong, CN); Kai Gao (Shandong, CN); Yaqian Zhao (Shandong, CN); Rengang Li (Shandong, CN)

Assignee: IEIT SYSTEMS CO., LTD.

H04L41/12G06F13/4009G06F13/4221G11B27/10G06F2213/0026

Abstract

Claims (59)

1. A data synchronization method, comprising:

constructing first level physical topologies of ring structures having a quantity consistent with a quantity of types of acceleration devices among acceleration devices of a same type in a target server, and constructing second level physical topologies of ring structures having a quantity consistent with a quantity of acceleration devices of the same type among different types of acceleration devices in the target server, wherein different types of acceleration devices supporting cache coherence protocol are provided in the target server, quantities of acceleration devices of different types are the same and at least two, and acceleration devices in the second level physical topologies are connected through the cache coherence protocol;

performing a first processing on to be synchronized data related to model training in acceleration devices of the same type through scatter reduce communication based on the first level physical topologies, and performing a second processing on data after the first processing in different types of acceleration devices through scatter reduce communication based on the second level physical topologies; and

performing a third processing on data after the second processing in different types of acceleration devices through all gather communication based on the second level physical topologies, and performing a fourth processing on data after the third processing in acceleration devices of the same type through all gather communication based on the first level physical topologies;

wherein a physical connection between different types of acceleration devices comprises a connection using central processing unit (CPU) as an intermediate medium and a connection using the cache coherence protocol; and

before the constructing the first level physical topologies of ring structures having the quantity consistent with the quantity of types of acceleration devices among acceleration devices of the same type in the target server, and constructing the second level physical topologies of ring structures having the quantity consistent with the quantity of acceleration devices of the same type among different types of acceleration devices in the target server, the method further comprises:

determining whether a bandwidth of data transmission between different types of acceleration devices through the connection using CPU as the intermediate medium is higher than a bandwidth of data transmission between different types of acceleration devices through the connection using the cache coherence protocol; and

using the connection using the cache coherence protocol as the physical connection between different types of acceleration devices when the bandwidth of data transmission between different types of acceleration devices through the connection using CPU as the intermediate medium is higher than the bandwidth of data transmission between different types of acceleration devices through the connection using the cache coherence protocol.

2. The data synchronization method according to claim 1 , wherein acceleration devices corresponding to each of the first level physical topologies are independent of each other, and acceleration devices corresponding to different first level physical topologies perform concurrently when the first processing and the fourth processing are performed;

acceleration devices corresponding to each of the second level physical topologies are independent of each other, and acceleration devices corresponding to different second level physical topologies perform concurrently when the second processing and the third processing are performed.

3. The data synchronization method according to claim 1 , wherein before the constructing the first level physical topologies of ring structures having the quantity consistent with the quantity of types of acceleration devices among acceleration devices of the same type in the target server, and constructing the second level physical topologies of ring structures having the quantity consistent with the quantity of acceleration devices of the same type among different types of acceleration devices in the target server, the method further comprises:

setting the acceleration devices of different types with the quantity being the same and at least two in the target server, and connecting the acceleration devices of different types using the cache coherence protocol.

4. The data synchronization method according to claim 3 , wherein the setting the acceleration devices of different types with the quantity being the same and at least two in the target server, and connecting the acceleration devices of different types using the cache coherence protocol, comprises:

setting at least two graphics processing unit (GPU) devices and setting field programmable gate array (FPGA) devices with a quantity the same as the GPU devices in the target server, and connecting the GPU devices and the FPGA devices using the cache coherence protocol.

5. The data synchronization method according to claim 3 , wherein the setting the acceleration devices of different types with the quantity being the same and at least two in the target server, and connecting the acceleration devices of different types using the cache coherence protocol, comprises:

acquiring one acceleration device from different types of acceleration devices in the target server in sequence, and connecting a next acquired acceleration device to a last acquired acceleration device using the cache coherence protocol.

6. The data synchronization method according to claim 1 , wherein the performing the first processing on the to be synchronized data related to model training in acceleration devices of the same type through scatter reduce communication based on the first level physical topologies comprises:

performing the first processing on to be synchronized data related to model training in each acceleration device in the first level physical topologies through scatter reduce communication, to obtain at least one partial aggregation result of the to be synchronized data related to the model training, wherein the partial aggregation result in each acceleration device is different from partial aggregation results in other acceleration devices in the first level physical topologies.

7. The data synchronization method according to claim 1 , wherein the method further comprises:

performing the first processing on to be synchronized data related to model training in acceleration devices of the same type in each target server through scatter reduce communication based on the first level physical topologies corresponding to each target server when a plurality of target servers are provided.

8. The data synchronization method according to claim 1 , wherein the method further comprises:

constructing third level physical topologies of ring structures having a quantity consistent with the quantity of the acceleration devices of the same type in each target server among the acceleration devices of the same type in different target servers when a plurality of target servers are provided, wherein each of third level physical topologies comprises acceleration devices with a quantity the same as the target servers and located in different target servers; and

after performing the second processing on data after the first processing in different types of acceleration devices through scatter reduce communication based on the second level physical topologies, the method further comprises:

performing a reduce operation on data after the second processing in acceleration devices in different target servers based on the third level physical topologies and broadcasting reduced data to each acceleration device, so as to determine the broadcast data as data after the second processing in each acceleration device.

9. The data synchronization method according to claim 8 , wherein the performing the reduce operation on data after the second processing in acceleration devices in different target servers based on the third level physical topologies and broadcasting the reduced data to each acceleration device, comprises:

receiving data after the second processing in the acceleration devices in different target servers by using a programmable switch, performing a reduce operation on received data based on the third level physical topologies, and broadcasting the reduced data to each acceleration device by using the programmable switch, wherein each target server is connected to a programmable switch.

10. The data synchronization method according to claim 8 , wherein the constructing the first level physical topologies of ring structures having the quantity consistent with the quantity of types of acceleration devices among acceleration devices of the same type in the target server, and constructing the second level physical topologies of ring structures having a quantity consistent with the quantity of acceleration devices of the same type among different types of acceleration devices in the target server, comprises:

constructing the first level physical topologies corresponding to each target server among the acceleration devices of the same type in each target server, and constructing the second level physical topologies corresponding to each target server among the acceleration devices of different types in each target server; and

the performing the third processing on data after the second processing in different types of acceleration devices through all gather communication based on the second level physical topologies, and performing the fourth processing on data after the third processing in acceleration devices of the same type through all gather communication based on the first level physical topologies, comprises:

performing the third processing on data after the second processing in different types of acceleration devices in each target server through all gather communication based on the second level physical topologies corresponding to each target server, and performing the fourth processing on data after the third processing in acceleration devices of the same type in each target server through all gather communication based on the first level physical topologies corresponding to each target server.

11. The data synchronization method according to claim 8 , wherein acceleration devices corresponding to each of the third level physical topologies are independent of each other, and acceleration devices corresponding to different third level physical topologies perform concurrently when the reduce operation is performed.

12. The data synchronization method according to claim 1 , wherein before the constructing the first level physical topologies of ring structures having the quantity consistent with the quantity of types of acceleration devices among acceleration devices of the same type in the target server, the method further comprises:

determining whether a bandwidth of data transmission between acceleration devices of the same type through other available connection is higher than a bandwidth of data transmission through the connection using the cache coherence protocol; and

constructing physical topologies among the acceleration devices of the same type through the other available connection when the bandwidth of data transmission between the acceleration devices of the same type through the other available connection is higher than the bandwidth of data transmission through the connection using the cache coherence protocol.

13. The data synchronization method according to claim 12 , wherein the other available connection comprises an original connection; and

determining whether the bandwidth of data transmission between acceleration devices of the same type through the other available connection is higher than the bandwidth of data transmission through the connection using the cache coherence protocol; and

constructing physical topologies among the acceleration devices of the same type through the other available connection when the bandwidth of data transmission between the acceleration devices of the same type through the other available connection is higher than the bandwidth of data transmission through the connection using the cache coherence protocol, comprises:

determining whether a bandwidth of data transmission between acceleration devices of the same type through the original connection is higher than the bandwidth of data transmission through the connection using the cache coherence protocol; and

connecting the acceleration devices of the same type in pairs through the original connection when the bandwidth of data transmission between the acceleration devices of the same type through the original connection is higher than the bandwidth of data transmission through the connection using the cache coherence protocol.

14. The data synchronization method according to claim 13 , wherein the original connection comprises NVLink (NVIDIA Link, a bus and its communication protocol developed and introduced by NVIDIA); and

the determining whether the bandwidth of data transmission between acceleration devices of the same type through the original connection is higher than the bandwidth of data transmission through the connection using the cache coherence protocol, and connecting the acceleration devices of the same type in pairs through the original connection when the bandwidth of data transmission between acceleration devices of the same type through the original connection is higher than the bandwidth of data transmission through the connection using the cache coherence protocol, comprises:

determining whether a bandwidth of data transmission between acceleration devices of the same type through NVLink is higher than the bandwidth of data transmission through the connection using the cache coherence protocol; and

connecting the acceleration devices of the same type in pairs through NVLink when the bandwidth of data transmission between acceleration devices of the same type through NVLink is higher than the bandwidth of data transmission through the connection using the cache coherence protocol.

15. A data synchronization system, wherein the data synchronization system comprises a server cluster, servers in the server cluster comprise different types of acceleration devices supporting compute express link (CXL) protocol, the different types of acceleration devices are configured for deep neural network training; and

the data synchronization system is configured for constructing first level physical topologies of ring structures having a quantity consistent with a quantity of types of acceleration devices among acceleration devices of a same type in a target server, and constructing second level physical topologies of ring structures having a quantity consistent with a quantity of acceleration devices of the same type among different types of acceleration devices in the target server, wherein different types of acceleration devices supporting cache coherence protocol are provided in the target server, quantities of acceleration devices of different types are the same and at least two, and the acceleration devices in the second level physical topologies are connected through the cache coherence protocol;

performing a first processing on to be synchronized data related to model training in acceleration devices of the same type through scatter reduce communication based on the first level physical topologies, and performing a second processing on data after the first processing in different types of acceleration devices through scatter reduce communication based on the second level physical topologies; and

performing a third processing on data after the second processing in different types of acceleration devices through all gather communication based on the second level physical topologies, and performing a fourth processing on data after the third processing in the acceleration devices of the same type through all gather communication based on the first level physical topologies;

wherein a physical connection between different types of acceleration devices comprises a connection using central processing unit (CPU) as an intermediate medium and a connection using the cache coherence protocol; and

before the constructing the first level physical topologies of ring structures having the quantity consistent with the quantity of types of acceleration devices among acceleration devices of the same type in the target server, and constructing the second level physical topologies of ring structures having the quantity consistent with the quantity of acceleration devices of the same type among different types of acceleration devices in the target server, the method further comprises:

determining whether a bandwidth of data transmission between different types of acceleration devices through the connection using CPU as the intermediate medium is higher than a bandwidth of data transmission between different types of acceleration devices through the connection using the cache coherence protocol; and

using the connection using the cache coherence protocol as the physical connection between different types of acceleration devices when the bandwidth of data transmission between different types of acceleration devices through the connection using CPU as the intermediate medium is higher than the bandwidth of data transmission between different types of acceleration devices through the connection using the cache coherence protocol.

16. A non-transitory computer-readable storage medium, for storing a computer executable instruction, wherein the computer executable instruction, when loaded and executed by a processor, implements a data synchronization method according to claim 1 comprising:

constructing first level physical topologies of ring structures having a quantity consistent with a quantity of types of acceleration devices among acceleration devices of a same type in a target server, and constructing second level physical topologies of ring structures having a quantity consistent with a quantity of acceleration devices of the same type among different types of acceleration devices in the target server, wherein different types of acceleration devices supporting cache coherence protocol are provided in the target server, quantities of acceleration devices of different types are the same and at least two, and acceleration devices in the second level physical topologies are connected through the cache coherence protocol;

performing a first processing on to be synchronized data related to model training in acceleration devices of the same type through scatter reduce communication based on the first level physical topologies, and performing a second processing on data after the first processing in different types of acceleration devices through scatter reduce communication based on the second level physical topologies; and

performing a third processing on data after the second processing in different types of acceleration devices through all gather communication based on the second level physical topologies, and performing a fourth processing on data after the third processing in acceleration devices of the same type through all gather communication based on the first level physical topologies;

wherein a physical connection between different types of acceleration devices comprises a connection using central processing unit (CPU) as an intermediate medium and a connection using the cache coherence protocol; and

before the constructing the first level physical topologies of ring structures having the quantity consistent with the quantity of types of acceleration devices among acceleration devices of the same type in the target server, and constructing the second level physical topologies of ring structures having the quantity consistent with the quantity of acceleration devices of the same type among different types of acceleration devices in the target server, the method further comprises:

determining whether a bandwidth of data transmission between different types of acceleration devices through the connection using CPU as the intermediate medium is higher than a bandwidth of data transmission between different types of acceleration devices through the connection using the cache coherence protocol; and

using the connection using the cache coherence protocol as the physical connection between different types of acceleration devices when the bandwidth of data transmission between different types of acceleration devices through the connection using CPU as the intermediate medium is higher than the bandwidth of data transmission between different types of acceleration devices through the connection using the cache coherence protocol.

Assignments (3)

Priority Claims (1)

Continuity (1)

References Cited (19)