Hardware offload circuitry
Examples described herein relate to offload circuitry comprising one or more compute engines that are configurable to perform a workload offloaded from a process executed by a processor based on a descriptor particular to the workload. In some examples, the offload circuitry is configurable to perform the workload, among multiple different workloads. In some examples, the multiple different workloads include one or more of: data transformation (DT) for data format conversion, Locality Sensitive Hashing (LSH) for neural network (NN), similarity search, sparse general matrix-matrix multiplication (SpGEMM) acceleration of hash based sparse matrix multiplication, data encode, data decode, or embedding lookup.
1. An apparatus comprising:
offload circuitry comprising one or more compute engines that are configurable to perform a workload offloaded from a process executed by a processor based on a descriptor particular to the workload, wherein the offload circuitry is configurable to perform the workload, among multiple different workloads, and wherein the multiple different workloads comprises:
sparse general matrix-matrix multiplication (SpGEMM) acceleration of hash based sparse matrix multiplication, wherein for a workload comprising SpGEMM acceleration of hash based sparse matrix multiplication, for an operation of A×B=C, a descriptor format comprises one or more of: (1) metadata address of matrix A; (2) metadata address of matrix B; (3) address of hash table array to store a partial result; (4) start row identifier of matrix A to be processed; (5) end row identifier of matrix A to be processed; and/or (6) metadata address of matrix C.
2. The apparatus of claim 1 , wherein for a workload of the multiple different workloads that comprises data transformation (DT) for data format conversion, the descriptor format comprises: a pointer to a serialized string, address translation information, an address for an output, an address for a processing status, and/or a pointer to a transformation map.
3. The apparatus of claim 1 , wherein for a workload of the multiple different workloads that comprises data transformation (DT) for data format conversion, the one or more compute engines perform in parallel:
parse a transformation map to separate fields of data and provide the fields to processing queues for the one or more compute engines to perform data serialization or de-serialization.
4. The apparatus of claim 1 , wherein for a workload of the multiple different workloads comprising Locality Sensitive Hashing (LSH) for neural network (NN) or similarity search, the descriptor format comprises: an address of a queried key, addresses of hash tables, and/or address of a result buffer.
5. The apparatus of claim 1 , wherein for a workload of the multiple different workloads comprising Locality Sensitive Hashing (LSH) for neural network (NN) or similarity search, the one or more compute engines perform in parallel:
determine a subset of candidates and compute one or more distances and
perform candidate pruning to determine an integer K number of nearest neighbors.
6. The apparatus of claim 1 , wherein for the workload comprising SpGEMM acceleration of hash based sparse matrix multiplication, the one or more compute engines perform in parallel:
access a series of hash tables, walk the hash tables to determine an accumulated value, perform multiplication and accumulation on matrices, and provide a resulting output matrix.
7. The apparatus of claim 1 , wherein for a workload of the multiple different workloads comprising embedding lookup, the compute engines perform, in parallel, perform:
embedding lookup operations, floating-point multiplication and addition, with scaling, or multiply-accumulate (MAC) to compute a product of two numbers and add the product to an accumulator.
8. The apparatus of claim 1 , comprising the processor coupled to the offload circuitry, wherein the processor is to execute the process that offloaded the workload to the offload circuitry.
9. The apparatus of claim 8 , comprising a server, wherein the server comprises the offload circuitry and the processor.
10. The apparatus of claim 9 , comprising a datacenter, wherein the datacenter comprises the server and at least one memory pool and wherein the at least one memory pool comprises a second offload circuitry and a second processor.
11. The apparatus of claim 1 , wherein the multiple different workloads also comprise one or more of: data transformation (DT) for data format conversion, Locality Sensitive Hashing (LSH) for neural network (NN), similarity search, data encode, data decode, or embedding lookup.
12. At least one non-transitory computer-readable medium comprising instructions stored thereon, that if executed by one or more processors, cause the one or more processors to:
execute an operating system to provide capability to one or more processes to offload a workload to offload circuitry comprising one or more compute engines that are configurable to perform a workload offloaded from a process executed by a processor based on a descriptor particular to the workload, wherein the offload circuitry is configurable to perform the workload, among multiple different workloads, and wherein the multiple different workloads comprises:
sparse general matrix-matrix multiplication (SpGEMM) acceleration of hash based sparse matrix multiplication, wherein for a workload comprising SpGEMM acceleration of hash based sparse matrix multiplication, for an operation of A×B=C, a descriptor format comprises one or more of: (1) metadata address of matrix A; (2) metadata address of matrix B; (3) address of hash table array to store a partial result; (4) start row identifier of matrix A to be processed; (5) end row identifier of matrix A to be processed; and/or (6) metadata address of matrix C.
13. The computer-readable medium of claim 12 , wherein for a workload of the multiple different workloads that comprises data transformation (DT) for data format conversion, the descriptor format comprises: a pointer to a serialized string, address translation information, an address for an output, an address for a processing status, and/or a pointer to a transformation map.
14. The computer-readable medium of claim 12 , wherein for a workload of the multiple different workloads that comprises data transformation (DT) for data format conversion, the one or more compute engines perform in parallel:
parse a transformation map to separate fields of data and provide the fields to processing queues for the one or more compute engines to perform data serialization or de-serialization.
15. The computer-readable medium of claim 12 , wherein for a workload of the multiple different workloads comprising Locality Sensitive Hashing (LSH) for neural network (NN) or similarity search, the descriptor format comprises: an address of a queried key, addresses of hash tables, and/or address of a result buffer.
16. The computer-readable medium of claim 12 , wherein for a workload of the multiple different workloads comprising Locality Sensitive Hashing (LSH) for neural network (NN) or similarity search, the one or more compute engines perform in parallel:
determine a subset of candidates and compute one or more distances and
perform candidate pruning to determine an integer K number of nearest neighbors.
17. The computer-readable medium of claim 12 , wherein for the workload comprising SpGEMM acceleration of hash based sparse matrix multiplication, the one or more compute engines perform in parallel:
access a series of hash tables, walk the hash tables to determine an accumulated value, perform multiplication and accumulation on matrices, and provide a resulting output matrix.
18. The computer-readable medium of claim 12 , wherein for a workload of the multiple different workloads comprising embedding lookup, the compute engines perform, in parallel, perform:
embedding lookup operations, floating-point multiplication and addition, with scaling, or multiply-accumulate (MAC) to compute a product of two numbers and add the product to an accumulator.
19. The computer-readable medium of claim 12 , wherein the multiple different workloads also comprise one or more of: data transformation (DT) for data format conversion, Locality Sensitive Hashing (LSH) for neural network (NN), similarity search, data encode, data decode, or embedding lookup.
20. A method comprising:
offloading a workload to offload circuitry comprising one or more compute engines that are configurable to perform a workload offloaded from a process executed by a processor based on a descriptor particular to the workload, wherein the offload circuitry is configurable to perform the workload, among multiple different workloads, and wherein the multiple different workloads comprises:
sparse general matrix-matrix multiplication (SpGEMM) acceleration of hash based sparse matrix multiplication, wherein for a workload comprising SpGEMM acceleration of hash based sparse matrix multiplication, for an operation of A×B=C, a descriptor format comprises one or more of: (1) metadata address of matrix A; (2) metadata address of matrix B; (3) address of hash table array to store a partial result; (4) start row identifier of matrix A to be processed; (5) end row identifier of matrix A to be processed; and/or (6) metadata address of matrix C.
21. The method of claim 20 , wherein a descriptor associated with the workload configures the offload circuitry to perform the workload.
22. The method of claim 20 , wherein the multiple different workloads also comprise one or more of: data transformation (DT) for data format conversion, Locality Sensitive Hashing (LSH) for neural network (NN), similarity search, data encode, data decode, or embedding lookup.