Associative summing for high performance computing

1. Apparatus, comprising:

one or more communication interfaces for communicating over a communication network; and

a processor, configured to:

receive, via the communication interfaces, a first number and a second number, each of which is in a floating-point representation that includes a mantissa including M bits,

convert each of the first number and the second number to a derived floating-point representation that represents any given floating-point number by N signed integers corresponding to different respective orders of magnitude, each of the signed integers including a group of B magnitude bits, N being greater than or equal to (M−1)/B+1,

such that the first number is represented by N first signed integers, which include respective first groups of magnitude bits, and the second number is represented by N second signed integers, which include respective second groups of magnitude bits,

compute a partial sum in the derived floating-point representation, by adding the first groups of magnitude bits to the second groups of magnitude bits, respectively, using N sets of one or more overflow bits to store any sum of a first group and a second group that is greater than 2 B −1,

subsequently to computing the partial sum, receive, via the communication interfaces, one or more third numbers in the floating-point representation,

convert the one or more third numbers to the derived floating-point representation,

compute a final sum, by adding the one or more third numbers, in the derived floating-point representation, to the partial sum, and

output the final sum.

2. The apparatus according to claim 1 , wherein the apparatus is a network switch,

the communication interfaces being ports belonging to the network switch, and

the processor being a processor of the network switch.

3. The apparatus according to claim 1 , wherein the apparatus is a network interface controller (NIC),

the communication interfaces being ports belonging to the NIC, and

the processor being a processor of the NIC.

4. The apparatus according to claim 1 , wherein the processor is further configured to

convert the final sum from the derived floating-point representation to the floating-point representation, prior to

outputting the final sum.

5. The apparatus according to claim 1 , wherein B is a power of two.

6. The apparatus according to claim 1 , wherein the communication network includes a High Performance Computing (HPC) network, and wherein the numbers are respective partial results of a parallel computing task performed on the HPC network.

7. The apparatus according to claim 1 , wherein the first signed integers correspond to different respective first orders of magnitude, and wherein the processor is configured to convert the second number to the derived floating-point representation by:

representing the second number by N tentative signed integers corresponding to different respective second orders of magnitude,

computing a non-negative difference D between (i) a highest one of the first orders of magnitude, and (ii) a highest one of the second orders of magnitude, and

representing the second number by the N second signed integers, by shifting the tentative signed integers by D positions.

8. The apparatus according to claim 1 , wherein N is a smallest integer that is greater than or equal to (M−1)/B+1.

9. The apparatus according to claim 1 , wherein the processor is configured to iteratively (i) receive a respective one of the third numbers, (ii) convert the respective one of the third numbers to the derived floating-point representation, and (iii) update the partial sum, by adding the respective one of the third numbers, in the derived floating-point representation, to the partial sum.

10. A system, comprising:

a plurality of networked computers; and

at least one network switch, configured to:

receive, from the computers, a first number and a second number, each of which is in a floating-point representation that includes a mantissa including M bits,

convert each of the first number and the second number to a derived floating-point representation that represents any given floating-point number by N signed integers corresponding to different respective orders of magnitude, each of the signed integers including a group of B magnitude bits, N being greater than or equal to (M−1)/B+1,

such that the first number is represented by N first signed integers, which include respective first groups of magnitude bits, and the second number is represented by N second signed integers, which include respective second groups of magnitude bits,

compute a partial sum in the derived floating-point representation, by adding the first groups of magnitude bits to the second groups of magnitude bits, respectively, using N sets of one or more overflow bits to store any sum of a first group and a second group that is greater than 2 B −1,

subsequently to computing the partial sum, receive, from the computers, one or more third numbers in the floating-point representation,

convert the one or more third numbers to the derived floating-point representation,

compute a final sum, by adding the one or more third numbers, in the derived floating-point representation, to the partial sum, and

output the final sum.

11. The system according to claim 10 , wherein the network switch is further configured to:

convert the final sum from the derived floating-point representation to the floating-point representation, prior to

outputting the final sum.

12. The system according to claim 10 , wherein B is a power of two.

13. The system according to claim 10 , wherein the numbers are respective partial results of a parallel computing task performed by the computers.

14. The system according to claim 10 , wherein the first signed integers correspond to different respective first orders of magnitude, and wherein the network switch is configured to convert the second number to the derived floating-point representation by:

representing the second number by N tentative signed integers corresponding to different respective second orders of magnitude,

computing a non-negative difference D between (i) a highest one of the first orders of magnitude, and (ii) a highest one of the second orders of magnitude, and

representing the second number by the N second signed integers, by shifting the tentative signed integers by D positions.

15. The system according to claim 10 , wherein N is a smallest integer that is greater than or equal to (M−1)/B+1.

16. The system according to claim 10 , wherein the network switch is configured to iteratively (i) receive a respective one of the third numbers, (ii) convert the respective one of the third numbers to the derived floating-point representation, and (iii) update the partial sum, by adding the respective one of the third numbers, in the derived floating-point representation, to the partial sum.

17. A method, comprising:

receiving by a network switch, over a communication network, a first number and a second number, each of which is in a floating-point representation that includes a mantissa including M bits;

converting each of the first number and the second number to a derived floating-point representation that represents any given floating-point number by N signed integers corresponding to different respective orders of magnitude, each of the signed integers including a group of B magnitude bits, N being greater than or equal to (M−1)/B+1.

such that the first number is represented by N first signed integers, which include respective first groups of magnitude bits, and the second number is represented by N second signed integers, which include respective second groups of magnitude bits;

computing a partial sum in the derived floating-point representation, by adding the first groups of magnitude bits to the second groups of magnitude bits, respectively, using N sets of one or more overflow bits to store any sum of a first group and a second group that is greater than 2 B −1;

subsequently to computing the partial sum, receiving one or more third numbers in the floating-point representation;

converting the one or more third numbers to the derived floating-point representation;

computing a final sum, by adding the one or more third numbers, in the derived floating-point representation, to the partial sum; and

outputting the final sum.

18. The method according to claim 17 , comprising iteratively (i) receiving a respective one of the third numbers, (ii) converting the respective one of the third numbers to the derived floating-point representation, and (iii) updating the partial sum, by adding the respective one of the third numbers, in the derived floating-point representation, to the partial sum.

19. The method according to claim 17 , wherein the first signed integers correspond to different respective first orders of magnitude, and wherein converting the second number to the derived floating-point representation comprises:

representing the second number by N tentative signed integers corresponding to different respective second orders of magnitude,

computing a non-negative difference D between (i) a highest one of the first orders of magnitude, and (ii) a highest one of the second orders of magnitude, and

representing the second number by the N second signed integers, by shifting the tentative signed integers by D positions.

20. A computer software product comprising a tangible non-transitory computer-readable medium in which program instructions are stored, which instructions, when read by a processor, cause the processor to:

receive a first number and a second number, each of which is in a floating-point representation that includes a mantissa including M bits,

convert each of the first number and the second number to a derived floating-point representation that represents any given floating-point number by N signed integers corresponding to different respective orders of magnitude, each of the signed integers including a group of B magnitude bits, N being greater than or equal to (M−1)/B+1,

such that the first number is represented by N first signed integers, which include respective first groups of magnitude bits, and the second number is represented by N second signed integers, which include respective second groups of magnitude bits,

compute a partial sum in the derived floating-point representation, by adding the first groups of magnitude bits to the second groups of magnitude bits, respectively, using N sets of one or more overflow bits to store any sum of a first group and a second group that is greater than 2 B −1,

subsequently to computing the partial sum, receive, via the communication interfaces, one or more third numbers in the floating-point representation,

convert the one or more third numbers to the derived floating-point representation,

compute a final sum, by adding the one or more third numbers, in the derived floating-point representation, to the partial sum, and

output the final sum.