Systems and methods of instructions to accelerate multiplication of sparse matrices using bitmasks that identify non-zero elements
Disclosed embodiments relate to accelerating multiplication of sparse matrices. In one example, a processor is to fetch and decode an instruction having fields to specify locations of first, second, and third matrices, and an opcode indicating the processor is to multiply and accumulate matching non-zero (NZ) elements of the first and second matrices with corresponding elements of the third matrix, and executing the decoded instruction as per the opcode to generate NZ bitmasks for the first and second matrices, broadcast up to two NZ elements at a time from each row of the first matrix and each column of the second matrix to a processing engine (PE) grid, each PE to multiply and accumulate matching NZ elements of the first and second matrices with corresponding elements of the third matrix. Each PE further to store an NZ element for use in a subsequent multiplications.
1. An apparatus comprising:
a cache; and
a graphics processing unit coupled to the cache, wherein the graphics processing unit comprises:
a first register to store elements from a first matrix that has zero and non-zero values in a compressed format,
a second register to store elements from a second matrix,
a scheduler circuit to schedule an instruction for execution, the instruction comprising fields to specify the first register, the second register, an accumulation matrix, a destination matrix, indications of a logical matrix position of the elements in at least the first matrix in a non-compressed format, and an opcode to indicate the instruction is a sparse matrix instruction and that execution circuitry including a processing engine is to select a proper subset of elements of the second register from the second matrix as an input into a multiply-accumulator circuit of the processing engine based on the indications, multiply the elements from the first matrix with corresponding elements of the proper subset of elements of the second matrix to generate products, accumulate the products with corresponding elements of the accumulation matrix to produce sums, and store the sums in corresponding elements of the destination matrix, and
the execution circuitry, including the processing engine, to execute the instruction according to the opcode,
wherein the first matrix has M rows by K columns, the second matrix has K rows by N columns, the accumulation matrix has M rows by N columns, and the instruction includes a suffix to the opcode that when set to a first value is to explicitly specify a first set of K, M, and N values, and when set to a different second value is to explicitly specify a different second set of K, M, and N values.
2. The apparatus of claim 1 , wherein the processing engine comprises a multiplexer, and the multiplexer is to be controlled based on the indications of the logical matrix position of the elements in at least the first matrix in the non-compressed format to select the proper subset of elements of the second register from the second matrix as the input into the multiply-accumulator circuit.
3. The apparatus of claim 2 , wherein the graphics processing unit comprises a plurality of instances of the processing engine.
4. The apparatus of claim 1 , further comprising a central processing unit coupled to the graphics processing unit, wherein the graphics processing unit and the central processing unit are in a same package.
5. The apparatus of claim 1 , wherein the graphics processing unit supports multithreading.
6. The apparatus of claim 1 , wherein the elements from the first matrix are in the first register in row major layout.
7. The apparatus of claim 1 , wherein the instruction includes a second suffix to the opcode that when set to a first value is to explicitly specify the elements are floating-point format and when set to a different second value is to explicitly specify the elements are integer format.
8. The apparatus of claim 1 , wherein the execution circuitry further comprises an integer execution circuit and a floating-point execution circuit.
9. A method comprising:
storing elements from a first matrix that has zero and non-zero values in a compressed format in a first register of a graphics processing unit;
storing elements from a second matrix in a second register of the graphics processing unit;
scheduling, by a scheduler circuit of the graphics processing unit, an instruction for execution, the instruction comprising fields to specify the first register, the second register, an accumulation matrix, a destination matrix, indications of a logical matrix position of the elements in at least the first matrix in a non-compressed format, and an opcode to indicate the instruction is a sparse matrix instruction and that execution circuitry of the graphics processing unit including a processing engine is to select a proper subset of elements of the second register from the second matrix as an input into a multiply-accumulator circuit of the processing engine based on the indications, multiply the elements from the first matrix with corresponding elements of the proper subset of elements of the second matrix to generate products, accumulate the products with corresponding elements of the accumulation matrix to produce sums, and store the sums in corresponding elements of the destination matrix; and
executing, by the execution circuitry including the processing engine, the instruction according to the opcode,
wherein the first matrix has M rows by K columns, the second matrix has K rows by N columns, the accumulation matrix has M rows by N columns, and the instruction includes a suffix to the opcode that when set to a first value explicitly specifies a first set of K, M, and N values, and when set to a different second value explicitly specifies a different second set of K, M, and N values.
10. The method of claim 9 , wherein the processing engine comprises a multiplexer, and further comprising controlling the multiplexer based on the indications of the logical matrix position of the elements in at least the first matrix in the non-compressed format to select the proper subset of elements of the second register from the second matrix as the input into the multiply-accumulator circuit.
11. The method of claim 10 , wherein the graphics processing unit comprises a plurality of instances of the processing engine.
12. The method of claim 9 , further comprising sending data from the graphics processing unit to a central processing unit, wherein the graphics processing unit and the central processing unit are in a same package.
13. The method of claim 9 , wherein the graphics processing unit supports multithreading.
14. The method of claim 9 , wherein the elements from the first matrix are in the first register in row major layout.
15. The method of claim 9 , wherein the instruction includes a second suffix to the opcode that when set to a first value explicitly specifies the elements are floating-point format and when set to a different second value explicitly specifies the elements are integer format.
16. The method of claim 9 , wherein the execution circuitry further comprises an integer execution circuit and a floating-point execution circuit.
17. A non-transitory machine-readable medium containing code to which a machine is to respond by:
storing elements from a first matrix that has zero and non-zero values in a compressed format in a first register of a graphics processing unit;
storing elements from a second matrix in a second register of the graphics processing unit;
scheduling, by a scheduler circuit of the graphics processing unit, an instruction for execution, the instruction comprising fields to specify the first register, the second register, an accumulation matrix, a destination matrix, indications of a logical matrix position of the elements in at least the first matrix in a non-compressed format, and an opcode to indicate the instruction is a sparse matrix instruction and that execution circuitry of the graphics processing unit including a processing engine is to select a proper subset of elements of the second register from the second matrix as an input into a multiply-accumulator circuit of the processing engine based on the indications, multiply the elements from the first matrix with corresponding elements of the proper subset of elements of the second matrix to generate products, accumulate the products with corresponding elements of the accumulation matrix to produce sums, and store the sums in corresponding elements of the destination matrix; and
executing, by the execution circuitry including the processing engine, the instruction according to the opcode,
wherein the first matrix has M rows by K columns, the second matrix has K rows by N columns, the accumulation matrix has M rows by N columns, and the instruction includes a suffix to the opcode that when set to a first value explicitly specifies a first set of K, M, and N values, and when set to a different second value explicitly specifies a different second set of K, M, and N values.
18. The non-transitory machine-readable medium of claim 17 , wherein the processing engine comprises a multiplexer, and the machine is further to respond to the code by controlling the multiplexer based on the indications of the logical matrix position of the elements in at least the first matrix in the non-compressed format to select the proper subset of elements of the second register from the second matrix as the input into the multiply-accumulator circuit.
19. The non-transitory machine-readable medium of claim 18 , wherein the graphics processing unit comprises a plurality of instances of the processing engine.
20. The non-transitory machine-readable medium of claim 17 , wherein the instruction includes a second suffix to the opcode that when set to a first value explicitly specifies the elements are floating-point format and when set to a different second value explicitly specifies the elements are integer format.