Machine learning sparse computation mechanism
An apparatus to facilitate processing of a sparse matrix is disclosed. The apparatus includes a plurality of processing units each comprising one or more processing elements, including logic to read operands, a multiplication unit to multiply two or more operands and a scheduler to identify operands having a zero value and prevent scheduling of the operands having the zero value at the multiplication unit.
1. An apparatus comprising:
a graphics processing unit, including:
a plurality of processing units to perform neural network matrix multiplication operations, including the plurality of processing units to:
identify and track one or more operands having a zero value in a stored block of matrix data for a matrix;
identify and track one or more segments of sparse data in the stored block of matrix data, wherein identifying the one or more segments of sparse data includes performing pattern recognition on the stored block of matrix data;
schedule operands in the block of matrix data, including preventing scheduling of operands in the block of matrix data identified as having a zero value and of operands in the identified segments of sparse data;
read a plurality of operands from the block of matrix data; and
multiply two or more operands of the plurality of operands, wherein the multiplication of operands excludes the operands that are not scheduled.
2. The apparatus of claim 1 , wherein the plurality of processing units are to schedule non-zero operands for multiplication.
3. The apparatus of claim 2 , wherein the plurality of processing units are to retrieve stored operand values that are scheduled upon the operands being received.
4. The apparatus of claim 3 , wherein the plurality of processing units are to write a result of the multiplication of the two or more operands, wherein the plurality of processing units are to write a zero value for operands identified as having a zero value and operands in the identified segments of sparse data.
5. The apparatus of claim 1 , wherein performing pattern recognition includes the plurality of processing units to perform a bounding box operation on the block of matrix data to determine segments having a similarity of the data.
6. The apparatus of claim 5 , wherein the plurality of processing units are to track data stored at a memory device.
7. The apparatus of claim 5 , wherein the plurality of processing units are to track data stored at a cache memory device.
8. The apparatus of claim 5 , wherein the plurality of processing units are to track data stored at a page table.
9. The apparatus of claim 1 , wherein the plurality of processing units are further to record an address location for each identified segment of sparse data.
10. The apparatus of claim 1 , further comprising:
a sparse compressed buffer to store a compressed sparse matrix;
wherein the one or more processing units are to:
determine whether data of the matrix, including the identified one or more operands having a zero value and the identified one or more segments of sparse data, meets a threshold to qualify the matrix as a sparse matrix, and
upon determining that the matrix qualifies as a sparse matrix, compress the sparse matrix to generate a compressed sparse matrix and store the compressed sparse matrix in the sparse compressed buffer.
11. The apparatus of claim 10 , wherein the compressed sparse matrix is dynamically generated based on a sparse index.
12. The apparatus of claim 11 , wherein the compressed sparse matrix comprises a sparse matrix frequently processed at the plurality of processing units.
13. A graphics processing unit comprising:
a plurality of processing units to perform neural network matrix multiplication operations, including the plurality of processing units to:
identify and track one or more operands having a zero value in a stored block of matrix data for a matrix;
identify and track one or more segments of sparse data in a stored block of data, wherein identifying the one or more segments of sparse data includes performing pattern recognition on the stored block of matrix data;
determine whether data of the matrix, including the identified one or more operands having a zero value and the identified one or more segments of sparse data, meets a threshold to qualify the matrix as a sparse matrix, and
upon determining that the matrix qualifies as a sparse matrix, compress the sparse matrix to generate a compressed sparse matrix; and
a storage device to store the compressed sparse matrix.
14. The graphics processing unit of claim 13 , wherein the compressed sparse matrix is dynamically generated based on a sparse index.
15. The graphics processing unit of claim 13 , wherein the compressed sparse matrix comprises a sparse matrix frequently processed at the one or more processing units.
16. The graphics processing unit of claim 15 , wherein the plurality of processing units are further to record an address location for each identified segment of sparse data.
17. The graphics processing unit of claim 13 , wherein a bounding box operation on the block of matrix data to determine segments having a similarity of the data.
18. A method to perform neural network matrix multiplication operations comprising performing, by a graphics processing unit:
identifying and tracking one or more operands having a zero value in a stored block of matrix data for a matrix;
identifying and tracking one or more segments of sparse data in the stored block of matrix data, wherein identifying the one or more segments of sparse data includes performing pattern recognition on the stored block of matrix data;
scheduling operands in the block of matrix data, including preventing scheduling of operands in the block of matrix data identified as having a zero value and of operands in the identified segments of sparse data;
reading a plurality of operands from the block of matrix data; and
multiplying two or more operands of the plurality of operands, wherein the multiplication of operands excludes the operands that are not scheduled.
19. The method of claim 18 , wherein performing pattern recognition includes performing a bounding box operation on the block of matrix data to determine segments having a similarity of the data.
20. The method of claim 18 , further comprising:
determining whether data of the matrix, including the identified one or more operands having a zero value and the identified one or more segments of sparse data, meets a threshold to qualify the matrix as a sparse matrix, and
upon determining that the matrix qualifies as a sparse matrix, compressing the sparse matrix to generate a compressed sparse matrix and storing the compressed sparse matrix in a buffer.