Device and method for accelerating matrix multiply operations as a sum of outer products

1. A processing device comprising:

memory configured to store data; and

a processor comprising a plurality of processor cores which share the memory and are in communication with each other via first hierarchical communication links and second hierarchical communication links, each processor core in a group of processor cores being in communication with each other via the first hierarchical communication links and associated with a memory segment separate from the shared memory, and each processor core in the group of processor cores configured to:

store, in an associated memory segment separate from the shared memory, one of a plurality of sub-portions of data of a first matrix;

store, in the associated memory segment separate from the shared memory, one of a plurality of sub-portions of data of a second matrix;

determine an outer product of the sub-portion of data of the first matrix and the sub-portion of data of the second matrix;

receive, from another processor core of the group, another sub-portion of data of the second matrix;

store the other sub-portion of data of the second matrix in the associated memory segment that is separate from the shared memory; and

determine another outer product of the sub-portion of data of the first matrix and the other sub-portion of data of the second matrix.

2. The processing device according to claim 1 , wherein each of the plurality of processor cores is a compute unit comprising one or more single instruction, multiple data (SIMD) units.

3. The processing device according to claim 1 , wherein

the group of processor cores is one of a plurality of first hierarchical groups of processor cores;

the plurality of processor cores comprises a second hierarchical group of processor cores comprising the plurality of first hierarchical groups, the first hierarchical groups being in communication with each other via a plurality of second communication links,

each one of the first communication links extends a first distance, each one of the second communication links extends a second distance and the first distance is less than the second distance, and

the other sub-portion of data of the second matrix is received from the other processor core via one of the first communication links.

4. The processing device according to claim 3 , wherein

the sub-portions of data of the first matrix are sub-portions of one or more columns of the first matrix; and

the sub-portions of data of the second matrix are sub-portions of one or more rows of the second matrix.

5. The processing device according to claim 1 , wherein

each processor core is further configured to determine a partial outer product from the sub-portions of data of the first matrix and the sub-portions of data of the second matrix, and

the processor is configured to determine a third matrix from a sum of each partial outer product.

6. The processing device according to claim 1 , wherein each processor core is further configured to receive the other sub-portion of data of the second matrix from the other processor core without accessing the memory.

7. A processing device comprising:

memory configured to store data; and

a first plurality of processor cores which share the memory and are in communication with each other via first hierarchical communication links, the first plurality of processor cores comprising a first processor core and a second processor core each associated with a memory segment separate from the shared memory, wherein

the first processor core is configured to:

determine an outer product of a first sub-portion of data of a first matrix, which is stored in a first memory segment separate from the shared memory and associated with the first processor core, and a first sub-portion of data of a second matrix, which is stored in the first memory segment separate from the shared memory and associated with the first processor core; and

communicate, to the second processor core, via one of the first hierarchical communication links, the first sub-portion of data of the second matrix without communicating the first sub-portion of data of the first matrix to the second processor core; and

the second processor core is configured to:

receive the first sub-portion of data of the second matrix, communicated from the first processor core, without accessing the shared memory;

store the first sub-portion of data of the second matrix in a second memory segment separate from the shared memory and associated with the second processor core; and

determine another outer product of the first sub-portion of data of the second matrix received from the first processor core and a second sub-portion of data of the first matrix received from the memory.

8. The processing device according to claim 7 , wherein

the first processor core is configured to determine the outer product and communicate the first sub-portion of data of the second matrix at a first stage; and

the second processor core is configured to receive the first sub-portion of data of the second matrix and determine the other outer product at a second stage after the first stage.

9. The processing device according to claim 7 , wherein the first processor core is further configured to:

communicate the first sub-portion of data of the second matrix to the second processor core while the first sub-portion of data of the first matrix remains in the memory segment associated with the first processor core; and

the second processor core is further configured to communicate, to a third processor core via one of the first hierarchical communication links, the first sub-portion of data of the second matrix while the second sub-portion of data of the first matrix remains in a memory segment, separate from the memory, associated with the second processor core.

10. The processing device according to claim 7 , wherein each of the plurality of processor cores is a compute unit comprising one or more single instruction, multiple data (SIMD) units.

11. The processing device according to claim 7 , wherein

the plurality of processor cores is one of a plurality of first hierarchical groups of processor cores, each processor core of a first hierarchical group being in communication with each other via a plurality of the first communication links; and

the processing device further comprises a second hierarchical group of processor cores comprising the first hierarchical groups of processor cores, the first hierarchical groups being in communication with each other via a plurality of second communication links,

wherein each one of the first communication links extends a first distance, each one of the second communication links extends a second distance and the first distance is less than the second distance, and

the first sub-portion of data of the second matrix is communicated by the first processor core to the second processor core via one of the first communication links.

12. The processing device according to claim 7 , wherein

the first processor core is configured to receive remaining sub-portions of data of the second matrix from other processor cores of the plurality of processor cores and determine outer products for the remaining sub-portions of data of the second matrix and the first sub-portion of data of the first matrix while the first sub-portion of data of the first matrix remains in the memory segment, separate from the memory, associated with the first processor core, and

the second processor core is configured to receive the remaining sub-portions of data of the second matrix from the other processor cores of the plurality of processor cores and determine outer products for the remaining sub-portions of data of the second matrix and the second sub-portion of data of the first matrix while the second sub-portion of data of the first matrix remains in the memory segment, separate from the memory, associated with the second processor core.

13. The processing device according to claim 7 , wherein

the first sub-portion of data of the first matrix is a sub-portion of one or more columns of the first matrix; and

the first sub-portion of data of the second matrix is a sub-portion of one or more rows of the second matrix.

14. A method for use in a processing device having a plurality of processor cores, which share a memory and communicate with each other via first hierarchical communication links for performing matrix multiplication as a sum of partial outer products, the method comprising:

receiving, from the shared memory by a first processor core, a first sub-portion of data of a first matrix;

receiving, from the shared memory by the first processor core, a first sub-portion of data of a second matrix;

determining, by the first processor core, an outer product of the first sub-portion of data of the first matrix, which is stored in a first memory segment separate from the shared memory and associated with the first processor core and the first sub-portion of data of the second matrix, which is stored in the first memory segment separate from the shared memory and associated with the first processor core;

communicating, by the first processor core to a second processor core, the first sub-portion of data of the second matrix via one of the first hierarchical communication links;

receiving, from the shared memory by the second processor core, a second sub-portion of data of the first matrix;

storing the first sub-portion of data of the second matrix in a second memory segment separate from the shared memory and associated with the second processor core; and

determining, by the second processor core, another outer product of the second sub-portion of data of the first matrix received from memory and the first sub-portion of data of the second matrix communicated by the first processor core.

15. The method according to claim 14 , further comprising communicating, by the first processor core, the first sub-portion of data of the second matrix to the second processor core without accessing the memory.

16. The method according to claim 14 , further comprising communicating, by the second processor core, the first sub-portion of data of the second matrix to a third processor core.

17. The method according to claim 14 , further comprising:

determining, by the first processor core, outer products for the first sub-portion of data of the first matrix and remaining sub-portions of data of the second matrix by receiving each remaining sub-portion of data of the second matrix from other processor cores via the plurality of first hierarchical communication links while the first sub-portion of data of the first matrix remains in the memory segment separate from the memory and associated with the first processor core, and

determining, by the second processor core, outer products for the second sub-portion of data of the first matrix and the remaining sub-portions of data of the second matrix by receiving each remaining sub-portion of data of the second matrix from other processor cores via the plurality of first hierarchical communication links while the second sub-portion of data of the first matrix remains in the memory segment associated with the second processor core.

18. The method according to claim 14 , wherein

the first sub-portion of data of the first matrix is a sub-portion of one or more columns of the first matrix, and

the first sub-portion of data of the second matrix is a sub-portion of one or more rows of the second matrix.

19. The method according to claim 14 , wherein

the first processor core and the second processor core are cores of a first group of processor cores, the processor cores of the first group being in communication with each other via the plurality of first hierarchical communication links; and

the first group of processor cores is in communication with a second group of processor cores via one of a plurality of second hierarchical communication links,

wherein each one of the first hierarchical communication links extends a first distance, each one of the second communication hierarchical links extends a second distance and the first distance is less than the second distance.