Sparse matrix multiplication acceleration mechanism
An apparatus to facilitate acceleration of matrix multiplication operations. The apparatus comprises a systolic array including matrix multiplication hardware to perform multiply-add operations on received matrix data comprising data from a plurality of input matrices and sparse matrix acceleration hardware to detect zero values in the matrix data and perform one or more optimizations on the matrix data to reduce multiply-add operations to be performed by the matrix multiplication hardware.
1. An apparatus to facilitate acceleration of matrix multiplication operations, comprising:
a systolic array including matrix multiplication hardware to perform multiply-add operations on received matrix data comprising data from a plurality of input matrices; and
sparse matrix acceleration hardware to detect zero values in the matrix data and perform one or more optimizations on the matrix data to reduce multiply-add operations to be performed by the matrix multiplication hardware, including swapping rows in each of a plurality of sub-matrices of a first of the plurality of input matrices to achieve a maximum number of adjacent rows having a predetermined threshold zero values, wherein the swapping comprises swapping a first row having the predetermined threshold zero values with a second row not having the predetermined threshold zero values to enable the first row to be adjacent to a third row having the predetermined threshold zero values.
2. The apparatus of claim 1 , wherein the sparse matrix acceleration hardware detects zeroes within the received matrix data by comparing each matrix value with a zero value.
3. The apparatus of claim 1 , further comprising compression hardware to compress the matrix data.
4. The apparatus of claim 3 , wherein compressing the matrix data comprises generating packed matrix data by removing zero values from the matrix data and generating an indicator vector to identify the location of the zero values in the packed matrix data.
5. The apparatus of claim 4 , wherein the sparse matrix acceleration hardware receives the compressed matrix data and identifies the zero values in the compressed matrix data based on the indicator vector.
6. The apparatus of claim 1 , wherein the sparse matrix acceleration hardware optimizing the matrix data further comprises swapping rows in each of a plurality of sub-matrices of a second of the plurality of input matrices to add to results of a multiplication operation of the first matrix and a third of the plurality of input matrices.
7. The apparatus of claim 6 , wherein the sparse matrix acceleration hardware optimizing the matrix data further comprises performing a reverse swapping on an output matrix.
8. The apparatus of claim 6 , wherein the sparse matrix acceleration hardware optimizing the matrix data further comprises performing row adjustments in the first matrix.
9. The apparatus of claim 8 , wherein performing the row adjustments comprises shifting all non-zero values of a row having more than the predetermined number of zero values.
10. The apparatus of claim 8 , wherein performing the row adjustments comprises combining adjacent rows having more than the predetermined number of zero values.
11. The apparatus of claim 8 , wherein the matrix multiplication hardware comprises a systolic multiplier, including:
a first set of first in first out (FIFO) buffers to store data in the first input matrix;
a second set of FIFO buffers to store data in the second input matrix;
a plurality of processing elements (PEs), each coupled to receive data from at least one of the first set of FIFO buffers and at least one of the second set of FIFO buffers; and
a plurality of storage elements to locally store intermediate matrix multiplication values.
12. A method to facilitate acceleration of matrix multiplication operations, comprising:
detecting zero values in the matrix data comprising data from a plurality of input matrices;
performing one or more optimizations on the matrix data to eliminate a plurality of multiply-add operations that are to be performed on the matrix data, including swapping rows in each of a plurality of sub-matrices of a first of the plurality of input matrices to achieve a maximum number of adjacent rows having a predetermined threshold zero values, wherein the swapping comprises swapping a first row having the predetermined threshold zero values with a second row not having the predetermined threshold zero values to enable the first row to be adjacent to a third row having the predetermined threshold zero values; and
performing multiply-add operations on the optimized matrix data at matrix multiplication hardware.
13. The apparatus of claim 12 , wherein performing the one or more optimizations on the matrix data comprises swapping rows in each of a plurality of sub-matrices of a first of the plurality of input matrices to achieve a maximum number of adjacent rows having a predetermined threshold zero values.
14. The apparatus of claim 13 , wherein performing the one or more optimizations on the matrix data further comprises performing row adjustments in the first matrix.
15. The apparatus of claim 14 , wherein performing the one or more optimizations on the matrix data further comprises performing a reverse swapping on an output matrix resulting from the multiply-add operations.
16. A hardware accelerator comprising:
a systolic array, including:
matrix multiplication hardware to perform multiply-add operations on received matrix data comprising data from a plurality of input matrices; and
sparse matrix acceleration hardware to detect zero values in the matrix data and perform one or more optimizations on the matrix data to reduce multiply-add operations to be performed by the matrix multiplication hardware, including swapping rows in each of a plurality of sub-matrices of a first of the plurality of input matrices to achieve a maximum number of adjacent rows having a predetermined threshold zero values, wherein the swapping comprises swapping a first row having the predetermined threshold zero values with a second row not having the predetermined threshold zero values to enable the first row to be adjacent to a third row having the predetermined threshold zero values.
17. The accelerator of claim 16 , further comprising compression hardware to compress the matrix data.
18. The accelerator of claim 17 , wherein compressing the matrix data comprises generating packed matrix data by removing zero values from the matrix data and generating an indicator vector to identify the location of the zero values in the packed matrix data.
19. The accelerator of claim 18 , wherein the sparse matrix acceleration hardware receives the compressed matrix data and identifies the zero values in the compressed matrix data based on the indicator vector.