High performance computing system
A method including providing a SHARP tree including a plurality of data receiving processes and at least one aggregation node, designating a data movement command, providing a plurality of data input vectors to each of the plurality of data receiving processes, respectively, the plurality of data receiving processes each passing on the respective received data input vector to the at least one aggregation node, and the at least one aggregation node carrying out the data movement command on the received plurality of data input vectors. Related apparatus and methods are also provided.
1. A method for computation, comprising:
in a high-performance computing system that runs an application in which multiple processes perform portions of work of the application, defining a network comprising the multiple processes and at least one aggregation node;
in response to a data movement command, passing respective vectors of data having different, respective data sizes from the multiple processes over the network to the at least one aggregation node;
at the at least one aggregation node, in response to the data movement command, receiving and concatenating the respective vectors to generate a result vector having a result vector size equal to a sum of the respective data sizes of the received vectors; and
outputting the result vector from the at least one aggregation node to the network,
wherein defining the network comprises providing multiple aggregation nodes, comprising a root node, which outputs the result vector, and at least two intermediate aggregation nodes, each of which receives and concatenates the respective vectors from respective ones of the processes.
2. The method according to claim 1 , wherein the data movement command comprises a gather command.
3. The method according to claim 1 , wherein outputting the result vector comprises sending the result vector to the multiple processes.
4. The method according to claim 1 , wherein defining the network comprises providing a hierarchical tree having nodes corresponding to the multiple processes.
5. The method according to claim 1 , wherein passing the respective vectors comprises sending the data from the multiple processes with respective indexes, and wherein concatenating the respective vectors comprises adding the data to the result vector according to the respective indexes.
6. The method according to claim 5 , wherein sending the data comprises providing indications from the multiple processes to the at least one aggregation node of the respective data sizes of the respective vectors.
7. A high-performance computing system comprising multiple nodes, which are configured to run an application in which multiple processes perform portions of work of the application,
wherein a network comprising the multiple processes and at least one aggregation node is defined in the system, such that in response to a data movement command, the multiple processes pass respective vectors of data having different, respective data sizes over the network to the at least one aggregation node, and
wherein at the at least one aggregation node, in response to the data movement command, receives and concatenates the respective vectors to generate and outputs a result vector to the network having a result vector size equal to a sum of the respective data sizes of the received vectors,
wherein the network comprises multiple aggregation nodes, including a root node, which outputs the result vector, and at least two intermediate aggregation nodes, each of which receives and concatenates the respective vectors from respective ones of the processes.
8. The system according to claim 7 , wherein the data movement command comprises a gather command.
9. The system according to claim 7 , wherein the at least one aggregation node is configured to send the result vector to the multiple processes.
10. The system according to claim 7 , wherein the network comprises is defined as a hierarchical tree in which the nodes correspond to the multiple processes.
11. The system according to claim 7 , wherein the multiple processes pass the respective vectors to the at least one aggregation node together with respective indexes, and wherein the at least one aggregation node concatenates the respective vectors according to the respective indexes.
12. The system according to claim 11 , wherein the multiple processes provide indications to the at least one aggregation node of the respective data sizes of the respective vectors.