Sparse optimizations for a matrix accelerator architecture
Embodiments described herein include, software, firmware, and hardware logic that provides techniques to perform arithmetic on sparse data via a systolic processing unit. Embodiment described herein provided techniques to skip computational operations for zero filled matrices and sub-matrices. Embodiments additionally provide techniques to maintain data compression through to a processing unit. Embodiments additionally provide an architecture for a sparse aware logic unit.
1. A general purpose graphics processor comprising:
a processing resource including a matrix accelerator and a decoder, the matrix accelerator including a load filter to bypass a load of a sparse submatrix of an input matrix and the decoder to decode an encoded set of data associated with the input matrix to generate a decoded set of data, the decoder to decode the encoded set of data based on metadata associated with the encoded set of data, wherein the load filter is to bypass the load of the sparse submatrix based on the metadata associated with the encoded set of data,
the load filter configured to bypass a load of a near-sparse submatrix having a limited number of non-zero values; and
the matrix accelerator configured to send a message to indicate bypass of the near-sparse submatrix.
2. The general purpose graphics processor as in claim 1 , wherein the decoder is to provide the metadata associated with the encoded set of data to the load filter.
3. The general purpose graphics processor as in claim 2 , wherein the metadata associated with the encoded set of data includes a significance map, the significance map to indicate a zero or non-zero value for a bitstream of the decoded set of data.
4. The general purpose graphics processor as in claim 3 , wherein the decoder is included in the matrix accelerator and the matrix accelerator is to read one or more encoded submatrices as input.
5. The general purpose graphics processor as in claim 1 , wherein the matrix accelerator includes multiple processing elements.
6. The general purpose graphics processor as in claim 5 , wherein the multiple processing elements are arranged as a systolic array and configured to process one or more submatrices of a set of input matrix data.
7. The general purpose graphics processor as in claim 1 , wherein the processing resource additionally includes a hardware logic unit to perform an operation for a bypassed near-sparse submatrix of the input matrix.
8. A method comprising:
on a general-purpose graphics processing unit including a matrix accelerator:
tracking, via zero detection circuitry, zero-value operands of matrices to be input to a matrix accelerator;
bypassing a matrix multiply operation on the matrix accelerator in response to detection of a zero-value operand, the zero-value operand associated with a first submatrix of an input matrix, wherein the first submatrix is a zero-value submatrix;
bypassing a matrix multiply operation on the matrix accelerator in response to detection of a second submatrix of the input matrix, the second submatrix having a limited number of non-zero values; and
sending a message to a processing resource external to the matrix accelerator, the message to indicate bypass of the second submatrix.
9. The method as in claim 8 , wherein the input matrix is a zero-value matrix and all matrix multiply operations on the matrix accelerator are bypassed for the zero-value matrix.
10. The method as in claim 8 , additionally comprising:
receiving the message at the processing resource; and
configuring the processing resource to perform a bypassed operation for the second submatrix.
11. The method as in claim 10 , additionally comprising performing the bypassed operation for the second submatrix via the processing resource based on the message.
12. The method as in claim 11 , wherein the message identifies the bypassed operation and operands for the bypassed operation.
13. The method as in claim 8 , wherein tracking, via the zero detection circuitry, zero-value operands of matrices to be input to a matrix accelerator includes generating a submatrix map for a first input matrix, the submatrix map to identify a zero-value submatrix for the first input matrix.
14. A data processing system comprising:
a memory device; and
a graphics processor coupled with the memory device, the graphics processor comprising:
a processing resource including a matrix accelerator and a decoder, the matrix accelerator including a load filter to bypass a load of a sparse submatrix of an input matrix and the decoder to decode an encoded set of data associated with the input matrix to generate a decoded set of data, the decoder to decode the encoded set of data based on metadata associated with the encoded set of data;
wherein the load filter is to bypass the load of the sparse submatrix based on the metadata associated with the encoded set of data;
wherein the load filter is to bypass load of a near-sparse submatrix having a limited number of non-zero values; and
wherein the matrix accelerator is to send a message to indicate bypass of the near-sparse submatrix.
15. The data processing system as in claim 14 , wherein the decoder is to provide the metadata associated with the encoded set of data to the load filter.
16. The data processing system as in claim 15 , wherein the metadata associated with the encoded set of data includes a significance map, the significance map to indicate a zero or non-zero value for a bitstream of the decoded set of data.
17. The data processing system as in claim 16 , wherein the decoder is included in the matrix accelerator and the matrix accelerator is to read one or more encoded submatrices as input.
18. The data processing system as in claim 14 , wherein the matrix accelerator includes multiple processing elements.
19. The data processing system as in claim 18 , wherein the multiple processing elements are arranged as a systolic array and configured to process one or more submatrices of a set of input matrix data.
20. The data processing system as in claim 14 , wherein the processing resource additionally includes a hardware logic unit to perform an operation for a bypassed near-sparse submatrix of the input matrix.