Generalized acceleration of matrix multiply accumulate operations
A method, computer readable medium, and processor are disclosed for performing matrix multiply and accumulate (MMA) operations. The processor includes a datapath configured to execute the MMA operation to generate a plurality of elements of a result matrix at an output of the datapath. Each element of the result matrix is generated by calculating at least one dot product of corresponding pairs of vectors associated with matrix operands specified in an instruction for the MMA operation. A dot product operation includes the steps of: generating a plurality of partial products by multiplying each element of a first vector with a corresponding element of a second vector; aligning the plurality of partial products based on the exponents associated with each element of the first vector and each element of the second vector; and accumulating the plurality of aligned partial products into a result queue utilizing at least one adder.
1. A processor, comprising:
one or more arithmetic logic units (ALUs) to:
receive at least one instruction to perform a matrix multiply and accumulate (MMA) operation; and
load, from a register file, data into logic circuitry to cause the logic circuitry to use the same data for multiple sub-operations of the MMA operation.
2. The processor of claim 1 , wherein the at least one instruction involves performing the MMA operation using elements from corresponding vectors of a first matrix and a second matrix.
3. The processor of claim 2 , wherein the same data from the first matrix is loaded into logic circuitry to cause the logic circuitry to use the same data for multiple sub-operations of the MMA operation involving the second matrix.
4. The processor of claim 1 , wherein sub-operations of the MMA operation include a series of dot product operations and addition operations.
5. The processor of claim 1 , wherein performing the MMA operation results in a resulting matrix, wherein each element of the resulting matrix is generated by calculating dot products of corresponding pairs of vectors from the first matrix and the second matrix in a single instruction cycle.
6. The processor of claim 5 , wherein the resulting matrix is generated without having to load new data from the register file in a subsequent instruction cycle.
7. The processor of claim 5 , wherein calculating dot products comprises:
generating a plurality of partial products by multiplying elements from the first matrix and corresponding elements of the second matrix; and
accumulating the plurality of partial products into the resulting matrix using an adder.
8. The processor of claim 1 , wherein the at least one instruction comprises multiple instructions.
9. A method, comprising:
receiving at least one instruction to perform an operation; and
loading, from a register file, data into logic circuitry to cause the logic circuitry to use the same data for multiple sub-operations of the operation.
10. The method of claim 9 , wherein the at least one instruction involves performing the operation using elements from corresponding vectors from a first matrix and a second matrix.
11. The method of claim 10 , wherein the same data from the first matrix is loaded into logic circuitry to cause the logic circuitry to use the same data for multiple sub-operations of the operation involving the second matrix.
12. The method of claim 10 , wherein the operation is a matrix multiply and accumulate (MMA) operation.
13. The method of claim 12 , wherein performing the MMA operation results in a resulting matrix, wherein each element of the resulting matrix is generated by calculating dot products of corresponding pairs of vectors from the first matrix and the second matrix in a single instruction cycle.
14. The method of claim 13 , wherein the resulting matrix is generated without having to load new data from the register file in a subsequent instruction cycle.
15. The method of claim 13 , wherein calculating dot products while performing the MMA operation comprises:
generating a plurality of partial products by multiplying elements from the first matrix and corresponding elements of the second matrix; and
accumulating the plurality of partial products into the resulting matrix using an adder.
16. A non-transitory, computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform steps comprising:
receive at least one instruction to perform an operation; and
load, from a register file, data into logic circuitry to cause the logic circuitry to use the same data for multiple sub-operations of the operation.
17. The non-transitory, computer-readable storage medium of claim 16 , wherein the at least one instruction is configured to perform the operation using elements from corresponding vectors of a first matrix and a second matrix.
18. The non-transitory, computer-readable storage medium of claim 17 , wherein the same data from the first matrix is loaded into logic circuitry to cause the logic circuitry to use the same data for multiple sub-operations of the operation involving the second matrix.
19. The non-transitory, computer-readable storage medium of claim 18 , wherein the multiple sub-operations include calculating dot products using elements from the first and second matrix.
20. The non-transitory, computer-readable storage medium of claim 19 , wherein the calculation of dot products includes:
generating a plurality of partial products by multiplying elements from the first matrix and corresponding elements of the second matrix; and
accumulating the plurality of partial products into a resulting matrix using an adder.