Interruptible and restartable matrix multiplication instructions, processors, methods, and systems
A processor of an aspect includes a decode unit to decode a matrix multiplication instruction. The matrix multiplication instruction is to indicate a first memory location of a first source matrix, is to indicate a second memory location of a second source matrix, and is to indicate a third memory location where a result matrix is to be stored. The processor also includes an execution unit coupled with the decode unit. The execution unit, in response to the matrix multiplication instruction, is to multiply a portion of the first and second source matrices prior to an interruption, and store a completion progress indicator in response to the interruption. The completion progress indicator to indicate an amount of progress in multiplying the first and second source matrices, and storing corresponding result data to the third memory location, that is to have been completed prior to the interruption.
1. A processor comprising:
a decoder to decode a matrix multiplication instruction having a first field associated with a first source matrix, a second field associated with a second source matrix, a destination field, and an opcode to identify the matrix multiplication instruction; and
an execution unit coupled to the decoder, the execution unit to perform operations in response to the matrix multiplication instruction, the operations including:
partitioning the first source matrix into a first plurality of tiles, each tile in the first plurality of tiles comprising a specified number of non-overlapping data elements, and
partitioning the second source matrix into a second plurality of tiles, each tile in the second plurality of tiles comprising a specified number of non-overlapping data elements,
the execution unit comprising a fused matrix multiplication and addition logic to perform parallel fused multiply-accumulate operations using data elements from a first tile of the first plurality of tiles and data elements from a second tile of the second plurality of tiles,
at least one of the parallel fused multiply-accumulate operations is to:
multiply data elements from the first tile and corresponding data elements from the second tile to generate a plurality of products, and add one or more of the plurality of products to a corresponding data element from an accumulation matrix to generate a corresponding result value in a result matrix.
2. The processor of claim 1 , wherein the first field is to indicate a location for the first source matrix, the second field is to indicate a location for the second source matrix, and the destination field is to indicate a location for the result matrix.
3. The processor of claim 1 , wherein the opcode corresponds to a size of the first source matrix, the second source matrix, and the result matrix.
4. The processor of claim 1 , wherein the fused matrix multiplication and addition logic comprises:
a plurality of multipliers, each multiplier to multiply one of the data elements from the first tile and one of the data elements from the second tile to generate one of the plurality of products, and
a plurality of adders, each adder to add one or more products of the plurality of products to the corresponding data element from the accumulation matrix to generate the corresponding result value in the result matrix.
5. The processor of claim 4 , wherein the decoder is to output a plurality of micro-operations or control signals responsive to the matrix multiplication instruction, a first plurality of the micro-operations or control signals indicating the multiplications to be performed by the plurality of multipliers.
6. The processor of claim 4 , wherein a size of the first tile and a size of the second tile are based on a number of the plurality of multipliers.
7. The processor of claim 4 , wherein the plurality of multipliers comprise a number of multipliers to optimize processing tiles of one or more predetermined sizes.
8. The processor of claim 7 , wherein the one or more predetermined sizes comprises tiles with 16 rows and 16 columns, tiles with 32 rows and 32 columns, or tiles with 64 rows and 64 columns.
9. A method comprising:
decoding a sequence of instructions, the sequence of instructions including a first matrix multiplication instruction having a first field associated with a first source matrix, a second field associated with a second source matrix, a destination field, and an opcode to identify the first matrix multiplication instruction in the sequence of instructions;
performing operations by an execution unit based on one or more instructions in the sequence of instructions;
partitioning, by a matrix multiplication accelerator responsive to the first matrix multiplication instruction, the first source matrix comprising a first plurality of data elements into a first plurality of tiles, each tile in the first plurality of tiles comprising a specified number of non-overlapping data elements from the first plurality of data elements, and to partition the second source matrix comprising a second plurality of data elements into a second plurality of tiles, each tile in the second plurality of tiles comprising a specified number of non-overlapping data elements from the second plurality of data elements;
performing parallel fused multiply-accumulate operations on a fused multiply-accumulate array using data elements from a first tile of the first plurality of tiles and data elements from a second tile of the second plurality of tiles, at least one fused multiply-accumulate operation to:
multiply data elements from the first tile and corresponding data elements from the second tile to generate a plurality of products, and
add one or more of the plurality of products to a corresponding data element from an accumulation matrix to generate a corresponding result value in a first result matrix.
10. The method of claim 9 wherein the first field is usable to identify a location for the first source matrix, the second field is usable to identify a location for the second source matrix, and the destination field is usable to identify a location for the first result matrix.
11. The method of claim 9 wherein the opcode corresponds to a size of the first source matrix, the second source matrix, and the first result matrix.
12. The method of claim 9 , wherein the fused multiply-accumulate array comprises:
a plurality of multipliers, each multiplier to multiply one of the data elements from the first tile and one of the data elements from the second tile to generate one of the plurality of products, and
a plurality of adders, each adder to add the one or more products of the plurality of products to the corresponding data element from the accumulation matrix to generate the corresponding result value in the first result matrix.
13. The method of claim 12 wherein decoding comprises outputting a plurality of micro-operations or control signals responsive to the first matrix multiplication instruction, a first plurality of the micro-operations or control signals indicating the multiplications to be performed by the plurality of multipliers.
14. The method of claim 12 wherein a size of the first tile and a size of the second tile are based on a number of the plurality of multipliers.
15. The method of claim 12 wherein the plurality of multipliers comprise a number of multipliers to optimize processing of tiles of one or more predetermined sizes.
16. The method of claim 15 wherein the one or more predetermined sizes comprises tiles with 16 rows and 16 columns, tiles with 32 rows and 32 columns, or tiles with 64 rows and 64 columns.
17. A non-transitory machine-readable medium having program code stored thereon which, when executed by a machine, causes the machine to perform operations comprising:
decoding a sequence of instructions, the sequence of instructions including a first matrix multiplication instruction having a first field associated with a first source matrix, a second field associated with a second source matrix, a destination field, and an opcode to identify the first matrix multiplication instruction in the sequence of instructions;
performing operations by an execution unit based on one or more instructions in the sequence of instructions;
partitioning, by a matrix multiplication accelerator responsive to the first matrix multiplication instruction, the first source matrix comprising a first plurality of data elements into a first plurality of tiles, each tile in the first plurality of tiles comprising a specified number of non-overlapping data elements from the first plurality of data elements, and to partition the second source matrix comprising a second plurality of data elements into a second plurality of tiles, each tile in the second plurality of tiles comprising a specified number of non-overlapping data elements from the second plurality of data elements;
performing parallel fused multiply-accumulate operations on a fused multiply-accumulate array using data elements from a first tile of the first plurality of tiles and data elements from a second tile of the second plurality of tiles, at least one fused multiply-accumulate operation to:
multiply data elements from the first tile and corresponding data elements from the second tile to generate a plurality of products, and
add one or more of the plurality of products to a corresponding data element from an accumulation matrix to generate a corresponding result value in a first result matrix.
18. The non-transitory machine-readable medium of claim 17 wherein the first field is usable to identify a location for the first source matrix, the second field is usable to identify a location for the second source matrix, and the destination field is usable to identify a location for the first result matrix.
19. The non-transitory machine-readable medium of claim 17 wherein the opcode corresponds to a size of the first source matrix, the second source matrix, and the first result matrix.
20. The non-transitory machine-readable medium of claim 17 , wherein the fused multiply-accumulate array comprises:
a plurality of multipliers, each multiplier to multiply one of the data elements from the first tile and one of the data elements from the second tile to generate one of the plurality of products, and
a plurality of adders, each adder to add the one or more products of the plurality of products to the corresponding data element from the accumulation matrix to generate the corresponding result value in the first result matrix.
21. The non-transitory machine-readable medium of claim 20 wherein decoding comprises outputting a plurality of micro-operations or control signals responsive to the first matrix multiplication instruction, a first plurality of the micro-operations or control signals indicating the multiplications to be performed by the plurality of multipliers.
22. The non-transitory machine-readable medium of claim 20 wherein a size of the first tile and a size of the second tile are based on a number of the plurality of multipliers.
23. The non-transitory machine-readable medium of claim 20 wherein the plurality of multipliers comprise a number of multipliers to optimize processing of tiles of one or more predetermined sizes.
24. The non-transitory machine-readable medium of claim 23 wherein the one or more predetermined sizes comprises tiles with 16 rows and 16 columns, tiles with 32 rows and 32 columns, or tiles with 64 rows and 64 columns.