Accelerator for sparse-dense matrix multiplication
Disclosed embodiments relate to an accelerator for sparse-dense matrix instructions. In one example, a processor to execute a sparse-dense matrix multiplication instruction, includes fetch circuitry to fetch the sparse-dense matrix multiplication instruction having fields to specify an opcode, a dense output matrix, a dense source matrix, and a sparse source matrix having a sparsity of non-zero elements, the sparsity being less than one, decode circuitry to decode the fetched sparse-dense matrix multiplication instruction, execution circuitry to execute the decoded sparse-dense matrix multiplication instruction to, for each non-zero element at row M and column K of the specified sparse source matrix generate a product of the non-zero element and each corresponding dense element at row K and column N of the specified dense source matrix, and generate an accumulated sum of each generated product and a previous value of a corresponding output element at row M and column N of the specified dense output matrix.
1. A system comprising:
a memory,
a cache configured to store data; and
a plurality of cores coupled to the cache, a core of the plurality of cores comprising:
execution circuitry configured to perform multiply-accumulate operations with a first source matrix and a second source matrix to generate a result matrix responsive to an instruction, wherein the first source matrix is a sparse matrix having non-zero data elements located at certain positions and is stored in a compressed format that identifies the positions of the non-zero data elements in the first source matrix, the execution circuitry further comprising:
a plurality of multiply-accumulate circuits configured to perform the multiply-accumulate operations, wherein the multiply-accumulate operations comprise a plurality of fused multiply-add operations to multiply the non-zero data elements of the first source matrix by corresponding data elements of the second source matrix identified based on the positions in the compressed format to generate a plurality of products, and to add the plurality of products to accumulated values to generate data elements of the result matrix.
2. The system of claim 1 , wherein the execution circuitry is further configured to use each position of a non-zero data element in the first source matrix to identify a row and/or column in the second source matrix, wherein the plurality of multiply-accumulate circuits multiply the non-zero data element in the first source matrix by each data element in the identified row and/or column in the second matrix, respectively.
3. The system of claim 1 , wherein the second source matrix comprises a dense matrix.
4. The system of claim 1 , wherein the compressed format comprises a compressed sparse row (CSR) format or a compressed sparse column (CSC) format.
5. The system of claim 1 , further comprising an instruction fetch circuit to fetch the instruction and a decoder to decode the instruction.
6. The system of claim 1 , wherein the instruction comprises a plurality of fields including a first field to specify an opcode, a second field to identify the result matrix, a third field to identify the first source matrix, and a fourth field to identify the second source matrix.
7. The system of claim 1 , wherein the first source matrix and the second source matrix comprise a machine learning activation matrix and/or a weight vector.
8. A method comprising:
storing, in a memory location, a first source matrix, which comprises a sparse matrix having non-zero data elements located at certain positions, in a compressed format that identifies the positions of the non-zero data elements in the first matrix; and
performing, via execution circuitry, multiply-accumulate operations with the first source matrix and a second source matrix to generate a result matrix responsive to executing an instruction, wherein performing the multiply-accumulate operations comprises:
multiplying, via multiply-accumulation circuits of the execution circuitry, the non-zero data elements of the first source matrix by corresponding data elements of the second source matrix identified based on the positions in the compressed format to generate a plurality of products; and
adding, via the multiply-accumulation circuits, the plurality of products to accumulated values to generate data elements of the result matrix.
9. The method of claim 8 , further comprising:
identifying, for each position of a non-zero data element in the first source matrix, a row and/or column in the second source matrix; and
multiplying the non-zero data element in the first source matrix by each data element in the identified row and/or column in the second matrix, respectively.
10. The method of claim 8 , wherein the second source matrix comprises a dense matrix.
11. The method of claim 8 , wherein the compressed format comprises a compressed sparse row (CSR) format or a compressed sparse column (CSC) format.
12. The method of claim 8 , further comprising:
fetching the instruction; and
decoding the instruction.
13. The method of claim 8 , wherein the instruction comprises a plurality of fields including a first field to specify an opcode, a second field to identify the result matrix, a third field to identify the first source matrix, and a fourth field to identify the second source matrix.
14. The method of claim 8 , wherein the first source matrix and the second source matrix comprise a machine learning activation matrix and/or a weight vector.
15. A non-transitory machine-readable medium having program code stored thereon which, when executed by a machine, causes the machine to perform operations of:
storing, in a memory location, a first source matrix, which comprises a sparse matrix having non-zero data elements located at certain positions, in a compressed format that identifies the positions of the non-zero data elements in the first matrix; and
performing, via execution circuitry, multiply-accumulate operations with the first source matrix and a second source matrix to generate a result matrix responsive to executing an instruction, wherein performing the multiply-accumulate operations comprises:
multiplying, via multiply-accumulation circuits of the execution circuitry, the non-zero data elements of the first source matrix by corresponding data elements of the second source matrix identified based on the positions in the compressed format to generate a plurality of products; and
adding, via the multiply-accumulation circuits, the plurality of products to accumulated values to generate data elements of the result matrix.
16. The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise:
identifying, for each position of a non-zero data element in the first source matrix, a row and/or column in the second source matrix; and
multiplying the non-zero data element in the first source matrix by each data element in the identified row and/or column in the second matrix, respectively.
17. The non-transitory machine-readable medium of claim 15 , wherein the second source matrix comprises a dense matrix.
18. The non-transitory machine-readable medium of claim 15 , wherein the compressed format comprises a compressed sparse row (CSR) format or a compressed sparse column (CSC) format.
19. The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise:
fetching the instruction; and
decoding the instruction.
20. The non-transitory machine-readable medium of claim 15 , wherein the instruction comprises a plurality of fields including a first field to specify an opcode, a second field to identify the result matrix, a third field to identify the first source matrix, and a fourth field to identify the second source matrix.
21. The non-transitory machine-readable medium of claim 15 , wherein the first source matrix and the second source matrix comprise a machine learning activation matrix and/or a weight vector.