Reducing bandwidth requirements for matrix multiplication
View Patent ↗A block matrix multiplication mechanism is provided for reversing the visitation order of blocks at corner turns when performing a block matrix multiplication operation in a data processing system. The mechanism increases block size and divides each block into sub-blocks. By reversing the visitation order, the mechanism eliminates a sub-block load at the corner turns. The mechanism performs sub-block matrix multiplication for each sub-block in a given block, and then repeats operation for a next block until all blocks are computed. The mechanism may determine block size and sub-block size to optimize load balancing and memory bandwidth. Therefore, the mechanism reduces maximum throughput and increases performance. In addition, the mechanism also reduces the number of multi-buffered local store buffers.
1. A method, in a data processing system, for optimized corner turns for local storage and bandwidth reduction, the method comprising:
for each block in a column of a block matrix multiplication operation:
dividing the block into sub-blocks;
loading one sub-block of a first block matrix or one sub-block of a second block matrix from a system memory into a local store;
loading one sub-block of a third block matrix from the system memory to the local store;
performing a sub-block matrix multiplication operation on a sub-block of the first block matrix and a sub-block of the second block matrix to form a product;
subtracting the product from a sub-block of the third block matrix to form a result sub-block of the third block matrix;
storing a result sub-block of the third block matrix from the local store to the system memory; and
repeating the sub-block matrix multiplication operation in a next column or row with a reversed visitation order until each result sub-block of a current block of the third block matrix has been stored to complete a block matrix multiplication operation for the current block of the third block matrix;
repeating the block matrix multiplication operation in a next column or row with a reversed visitation order.
2. The method of claim 1 , further comprising:
employing multi-buffering to hide data transfer latency.
3. The method of claim 2 , wherein employing multi-buffering comprises allocating two local store buffers for the first block matrix, the second block matrix, and the third block matrix.
4. The method of claim 1 , wherein performing a sub-block matrix multiplication operation comprises performing the sub-block matrix multiplication operation on a sub-block of the first block matrix and a sub-block of the second block matrix loaded in a previous time step.
5. The method of claim 1 , wherein storing a result sub-block comprises storing a result sub-block of the third block matrix formed in a previous time step.
6. The method of claim 1 , wherein repeating the sub-block matrix multiplication operation in a next column or row with a reversed visitation order comprises reusing a sub-block of the first block matrix or a sub-block of the second block matrix loaded in a previous time step.
7. The method of claim 1 , wherein dividing the block into sub-blocks comprises determining a sub-block size to optimize memory bandwidth and load balancing.
8. The method of claim 1 , wherein dividing the block into sub-blocks comprises short sub-blocks of the first block matrix and narrow sub-blocks of the second block matrix.
9. A computer program product comprising a computer recordable medium having'a computer readable program recorded thereon, wherein the computer readable program, when executed on a computing device, causes the computing device to:
for each block in a column of a block matrix multiplication operation:
divide the block into sub-blocks;
load one sub-block of a first block matrix or one sub-block of a second block matrix from a system memory into a local store;
load one sub-block of a third block matrix from the system memory to the local store;
perform a sub-block matrix multiplication operation on a sub-block of the first block matrix and a sub-block of the second block matrix to form a product;
subtract the product from a sub-block of the third block matrix to form a result sub-block of the third block matrix;
store a result sub-block of the third block matrix from the local store to the system memory; and
repeat the sub-block matrix multiplication operation in a next column or row with a reversed visitation order until each result sub-block of a current block of the third block matrix has been stored to complete a block matrix multiplication operation for the current block of the third block matrix;
repeat the block matrix multiplication operation in a next column or row with a reversed visitation order.
10. The computer program product of claim 9 , wherein loading one sub-block of a first block matrix or one sub-block of a second block matrix and loading one sub-block of a third block matrix from the system memory to the local store comprises employing double-buffering to hide data transfer latency.
11. The computer program product of claim 9 , wherein performing a sub-block matrix multiplication operation comprises performing the sub-block matrix multiplication operation on a sub-block of the first block matrix and a sub-block of the second block matrix loaded in a previous time step.
12. The computer program product of claim 9 , wherein repeating the sub-block matrix multiplication operation in a next column or row with a reversed visitation order comprises reusing a sub-block of the first block matrix or a sub-block of the second block matrix loaded in a previous time step.
13. The computer program product of claim 9 , wherein dividing the block into sub-blocks comprises determining a sub-block size to optimize memory bandwidth and load balancing.
14. The computer program product of claim 9 , wherein dividing the block into sub-blocks comprises short sub-blocks of the first block matrix and narrow sub-blocks of the second block matrix.
15. An apparatus, comprising:
a processor; and
a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to:
for each block in a column of a block matrix multiplication operation:
divide the block into sub-blocks;
load one sub-block of a first block matrix or one sub-block of a second block matrix from a system memory into a local store;
load one sub-block of a third block matrix from a system memory to the local store;
perform a sub-block matrix multiplication operation on a sub-block of the first block matrix and a sub-block of the second block matrix to form a product;
subtract the product from a sub-block of the third block matrix to form a result sub-block of the third block matrix;
store a result sub-block of the third block matrix from the local store to the system memory; and
repeat the sub-block matrix multiplication operation in a next column or row with a reversed visitation order until each result sub-block of a current block of the third block matrix has been stored to complete a block matrix multiplication operation for the current block of the third block matrix;
repeat the block matrix multiplication operation in a next column or row with a reversed visitation order.
16. The apparatus of claim 15 , wherein loading one sub-block of a first block matrix or one sub-block of a second block matrix and loading one sub-block of a third block matrix from the system memory to the local store comprises employing double-buffering to hide data transfer latency.
17. The apparatus of claim 15 , wherein performing a sub-block matrix multiplication operation comprises performing the sub-block matrix multiplication operation on a sub-block of the first block matrix and a sub-block of the second block matrix loaded in a previous time step.
18. The apparatus of claim 15 , wherein repeating the sub-block matrix multiplication operation in a next column or row with a reversed visitation order comprises reusing a sub-block of the first block matrix or a sub-block of the second block matrix loaded in a previous time step.
19. The apparatus of claim 15 , wherein dividing the block into sub-blocks comprises determining a sub-block size to optimize memory bandwidth and load balancing.
20. The apparatus of claim 15 , wherein dividing the block into sub-blocks comprises short sub-blocks of the first block matrix and narrow sub-blocks of the second block matrix.