Utilizing structured sparsity in systolic arrays
An apparatus to facilitate utilizing structured sparsity in systolic arrays is disclosed. The apparatus includes a processor comprising a systolic array to receive data from a plurality of source registers, the data comprising unpacked source data, structured source data that is packed based on sparsity, and metadata corresponding to the structured source data; identify portions of the unpacked source data to multiply with the structured source data, the portions of the unpacked source data identified based on the metadata; and output, to a destination register, a result of multiplication of the portions of the unpacked source data and the structured source data.
1. An apparatus comprising:
a processor comprising a systolic array to:
receive data from a plurality of source registers, the data comprising unpacked source data, structured source data that is packed based on sparsity, and metadata corresponding to the structured source data;
identify portions of the unpacked source data to multiply with the structured source data, the portions of the unpacked source data identified based on the metadata; and
output, to a destination register, a result of multiplication of the portions of the unpacked source data and the structured source data.
2. The apparatus of claim 1 , wherein the systolic array to perform dot product accumulate operations.
3. The apparatus of claim 1 , wherein the systolic array further comprises a plurality of multiplexor circuits to select the portions of the unpacked source data to multiply with the structured source data based on the metadata.
4. The apparatus of claim 1 , wherein the structured source data that is packed based on sparsity comprises elements of a broadcast register of the systolic array.
5. The apparatus of claim 1 , wherein the structured source data that is packed based on sparsity comprises elements of an index register of the systolic array.
6. The apparatus of claim 1 , wherein the systolic array to execute an instruction for sparse systolic dot product with accumulate to identify the portions of the unpacked source data to multiply with the structured source data using the metadata and to perform a dot product multiplication of the portions with the structured source data, and wherein the metadata is provided in a source register of the plurality of source registers called by the instruction.
7. The apparatus of claim 1 , wherein packed source data comprises at least one of a half-float datatype that packs two 16-bit elements into a channel, a bfloat datatype that packs two 16-bit elements into a channel, an int8 datatype that packs four 8-bit elements into a channel, an int4 datatype that packs eight 4-bit elements into a channel, or an int2 datatype that packs sixteen 2-bit elements into a channel.
8. The apparatus of claim 1 , wherein the metadata indicates a position of non-zero elements in an original form of the structured source data prior to packing into the structured source data.
9. The apparatus of claim 8 , wherein the original form of the structured source data is pre-processed by an external agent to pack into the structured source data by removing sparse elements from the original form, and wherein the external agent generates the metadata.
10. The apparatus of claim 9 , wherein the external agent comprises at least one of a central processing unit (CPU) or an intelligent sensor.
11. The apparatus of claim 1 , wherein the processor comprises a general-purpose graphics processing unit (GPGPU).
12. At least one non-transitory machine readable storage medium comprising instructions that, when executed, cause at least one processor to at least:
receiving, by a systolic array of the at least one processor, data from a plurality of source registers, the data comprising unpacked source data, structured source data that is packed based on sparsity, and metadata corresponding to the structured source data;
identifying portions of the unpacked source data to multiply with the structured source data, the portions of the unpacked source data identified based on the metadata; and
outputting, to a destination register, a result of multiplication of the portions of the unpacked source data and the structured source data.
13. The at least one non-transitory machine readable storage medium of claim 12 , wherein the systolic array to perform dot product accumulate operations.
14. The at least one non-transitory machine readable storage medium of claim 12 , wherein the systolic array further comprises a plurality of multiplexor circuits to select the portions of the unpacked source data to multiply with the structured source data based on the metadata.
15. The at least one non-transitory machine readable storage medium of claim 12 , wherein the systolic array to execute an instruction for sparse systolic dot product with accumulate to identify the portions of the unpacked source data to multiply with the structured source data using the metadata and to perform a dot product multiplication of the portions with the structured source data, and wherein the metadata is provided in a source register of the plurality of source registers called by the instruction.
16. The at least one non-transitory machine readable storage medium of claim 12 , wherein the metadata indicates a position of non-zero elements in an original form of the structured source data prior to packing into the structured source data.
17. A method comprising:
receiving, by a systolic array of a processing device, data from a plurality of source registers of the processing device, the data comprising unpacked source data, structured source data that is packed based on sparsity, and metadata corresponding to the structured source data;
identifying portions of the unpacked source data to multiply with the structured source data, the portions of the unpacked source data identified based on the metadata; and
outputting, to a destination register of the processing device, a result of multiplication of the portions of the unpacked source data and the structured source data.
18. The method of claim 17 , wherein the systolic array to perform dot product accumulate operations.
19. The method of claim 17 , wherein the systolic array further comprises a plurality of multiplexor circuits to select the portions of the unpacked source data to multiply with the structured source data based on the metadata.
20. The method of claim 17 , wherein the systolic array to execute an instruction for sparse systolic dot product with accumulate to identify the portions of the unpacked source data to multiply with the structured source data using the metadata and to perform a dot product multiplication of the portions with the structured source data, and wherein the metadata is provided in a source register of the plurality of source registers called by the instruction.