Hardware accelerator architecture for processing very-sparse and hyper-sparse matrix data
An accelerator architecture for processing very-sparse and hyper-sparse matrix data is disclosed. A hardware accelerator comprises one or more tiles, each including a plurality of processing elements (PEs) and a data management unit (DMU). The PEs are to perform matrix operations involving very- or hyper-sparse matrices that are stored by a memory. The DMU is to provide the plurality of PEs access to the memory via an interface that is optimized to provide low-latency, parallel, random accesses to the memory. The PEs, via the DMU, perform the matrix operations by, issuing random access read requests for values of the one or more matrices, issuing random access read requests for values of one or more vectors serving as a second operand, and issuing random access write requests for values of one or more vectors serving as a result.
1. A hardware accelerator comprising:
one or more tiles, wherein each tile includes:
a plurality of processing elements (PEs) to perform matrix operations involving, as a first operand, one or more very- or hyper-sparse matrices that are stored by a memory; and
a data management unit (DMU) to provide the plurality of PEs access to the memory, the memory to be coupled with the hardware accelerator via an interface that is optimized to provide low-latency, parallel, random accesses to data;
wherein the plurality of PEs, via the DMU, perform the matrix operations by,
issuing a first set of random access read requests for values of the one or more matrices after identifying locations of the values by issuing random access read requests for pointer values;
issuing a second set of random access read requests for values of a first set of one or more vectors serving as a second operand; and
issuing a third set of random access write requests for values of a second set of one or more vectors serving as a result.
2. The hardware accelerator of claim 1 , wherein the DMU comprises a cache to store data returned responsive to the issued first set of random access read requests for values of the one or more matrices.
3. The hardware accelerator of claim 1 , wherein the memory is a system memory also utilized by a hardware processor.
4. The hardware accelerator of claim 1 , wherein the hardware accelerator is to perform the matrix operations responsive to an offload of one or more tasks issued by hardware processor.
5. The hardware accelerator of claim 1 , wherein the one or more matrices are stored in a compressed format.
6. The hardware accelerator of claim 1 , wherein the matrix operations include multiplication operations.
7. The hardware accelerator of claim 1 , wherein the matrix operations include scale and update operations, multiplication operations, and dot product operations.
8. A method in a hardware accelerator for performing matrix operations with very-sparse or hyper-sparse matrices comprising:
issuing, by one or more processing elements (PEs) of a plurality of PEs of one or more tiles, a first set of random access read requests via one or more data management units (DMUs) to a memory for values of one or more very-sparse or hyper-sparse matrices after identifying locations of the values by issuing random access read requests for pointer values, wherein the one or more DMUs access the memory via an interface that is optimized to provide low-latency, parallel, random accesses to data;
issuing, by the one or more PEs via the one or more DMUs, a second set of random access read requests for values of a first set of one or more vectors serving as an operand; and
issuing, by the one or more PEs via the one or more DMUs, a third set of random access write requests for values of a second set of one or more vectors serving as a result.
9. The method of claim 8 , wherein the DMU comprises a cache to store data returned responsive to the issued first set of random access read requests for values of the one or more matrices.
10. The method of claim 8 , wherein the memory is a system memory also utilized by a hardware processor.
11. The method of claim 8 , wherein the issuing the first set of requests, second set of requests, and third set of requests occurs responsive to an offload of one or more tasks by a hardware processor to the hardware accelerator.
12. The method of claim 8 , wherein the one or more matrices are stored in a compressed format.
13. The method of claim 8 , wherein the matrix operations include multiplication operations.
14. The method of claim 8 , wherein the matrix operations include scale and update operations, multiplication operations, and dot product operations.
15. A system comprising:
a memory;
one or more tiles, wherein each tile includes:
a plurality of processing elements (PEs) to perform matrix operations involving, as a first operand, one or more very- or hyper-sparse matrices that are stored by the memory; and
a data management unit (DMU) to provide the plurality of PEs access to the memory, the memory coupled with the hardware accelerator via an interface that is optimized to provide low-latency, parallel, random accesses to data;
wherein the plurality of PEs, via the DMU, perform the matrix operations by,
issuing a first set of random access read requests for values of the one or more matrices after identifying locations of the values by issuing random access read requests for pointer values;
issuing a second set of random access read requests for values of a first set of one or more vectors serving as a second operand; and
issuing a third set of random access write requests for values of a second set of one or more vectors serving as a result.
16. The system of claim 15 , wherein the DMU comprises a cache to store data returned responsive to the issued first set of random access read requests for values of the one or more matrices.
17. The system of claim 15 , wherein the memory is a system memory also utilized by a hardware processor.
18. The system of claim 15 , wherein the system is to perform the matrix operations responsive to an offload of one or more tasks issued by a hardware processor.
19. The system of claim 15 , wherein the one or more matrices are stored in a compressed format.
20. The system of claim 15 , wherein the matrix operations include scale and update operations, multiplication operations, and dot product operations.