Remote DMA systems and methods for supporting synchronization of distributed processes in a multi-processor system using collective operations
The invention relates to a remote DMA system, and methods for supporting synchronization of distributed processes in a multiprocessor system using collective operations. One aspect of the invention is a multi-node computer system having a plurality of interconnected processing nodes. This system uses DMA engines to perform collective operations synchronizing processes executing on a set of nodes. Each process in the set of processes causes the DMA engine on the node on which the process executes, to transmit a collective operation command to the master node when the process reaches a synchronization point in its execution. The DMA engine on the master node receives and executes the collective operations from the processes, and in response to receiving a pre-established number of the collective operations, conditionally executing the set of associated commands.
1 . In a multi-node computer system having a plurality of interconnected processing nodes, a method of using DMA engines to perform collective operations to synchronize processes executing on a set of nodes, the method comprising:
identifying a DMA engine on one of the nodes of the set of nodes as a master node;
associating a set of commands with a collective operation and providing the set of commands to the DMA engine of the master node;
each process in the set of processes causing the DMA engine on the node on which the process executes, to transmit a collective operation command to the master node when the process reaches a synchronization point in its execution;
the DMA engine on the master node receiving and executing the collective operations from the processes, and in response to receiving a pre-established number of the collective operations, conditionally executing the set of associated commands.
2 . The method of claim 1 , wherein the set of associated commands includes commands to inform processes of the synchronization event.
3 . The method of claim 1 , wherein the collective operation includes a counting command and wherein if the count equals a pre-established count, the DMA engine executes associated commands stored in a processor memory accessible by the DMA engine of the master node.
4 . The method of claim 3 , wherein the counting command uses a hardware counter located within the DMA engine.
5 . The method of claim 1 , wherein the plurality of interconnected processing nodes are interconnected in at least one of a Kautz and de Bruijn topology.
6 . The method of claim 1 , wherein the collective operation is a barrier operation.
7 . The method of claim 1 , wherein the collective operation is a reduction operation.
8 . In a multi-node computer system having a plurality of interconnected processing nodes, the system including a DMA engines capable of performing collective operations to synchronize processes executing on a set of nodes, the system comprising:
a master node with a DMA engine, and with a collective operation associated with a set of commands;
a set of processes executing on a set of nodes, wherein each process causes the DMA engine on the node on which the process executes, to transmit a collective operation command to the master node when the process reaches a synchronization point in its execution;
wherein the DMA engine on the master node receives and executes the collective operations from the processes, and in response to receiving a pre-established number of the collective operations, conditionally executes the set of associated commands.
9 . The system of claim 8 , wherein the set of associated commands includes commands to inform processes of the synchronization event.
10 . The system of claim 8 , wherein the collective operation includes a counting command and wherein if the count equals a pre-established count, the DMA engine executes associated commands stored in a processor memory accessible by the DMA engine of the master node.
11 . The system of claim 10 , wherein the counting command uses a hardware counter located within the DMA engine.
12 . The system of claim 8 , wherein the plurality of interconnected processing nodes are interconnected in at least one of a Kautz and de Bruijn topology.
13 . The system of claim 8 , wherein the collective operation is a barrier operation.
14 . The system of claim 8 , wherein the collective operation is a reduction operation.