Instructions for structured-sparse tile matrix FMA
Systems, methods, and apparatuses relating sparsity based FMA. In some examples, an instance of a single FMA instruction has one or more fields for an opcode, one or more fields to identify a source/destination matrix operand, one or more fields to identify a first plurality of source matrix operands, one or more fields to identify a second plurality of matrix operands, wherein the opcode is to indicate that execution circuitry is to select a proper subset of FP8 data elements from the first plurality of source matrix operands based on sparsity controls from a first matrix operand of the second plurality of matrix operands and perform a FMA.
1. An apparatus comprising:
decode circuitry to decode an instance of a single instruction having one or more fields for an opcode, one or more fields to identify a source/destination matrix operand, one or more fields to identify a first plurality of source matrix operands, one or more fields to identify a second plurality of matrix operands, wherein the opcode is to indicate that execution circuitry is to select a proper subset of 8-bit floating point (FP8) data elements from the first plurality of source matrix operands based on sparsity controls from a first matrix operand of the second plurality of matrix operands, for each element position of the source/destination matrix operand, convert pairs of FP8 elements, as selected, from a row of the first source matrix operands and pairs of FP8 elements from a column of one of the second source matrix operands to 32-bit floating point (FP32), multiply converted even FP8 elements from the identified source matrix operands to generate a first product and separately multiply converted odd FP8 elements from the specified source matrix operands to generate second product, and accumulate the first and second products with previous contents of the source/destination matrix operand, wherein to convert a FP8 element at least comprises applying an exponent variable bias, provided by an operand of the instruction, during a conversion; and
execution circuitry to respond to the decoded instruction as specified by the opcode.
2. The apparatus of claim 1 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating point format.
3. The apparatus of claim 2 , wherein the sparsity controls are to select four data elements from the first plurality of source matrix operands per row.
4. The apparatus of claim 1 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating-point format having 1 bit for a sign, 5 bits for an exponent, and two bits for a fraction.
5. The apparatus of claim 4 , wherein the sparsity controls are to select four data elements from the first plurality of source matrix operands per row.
6. The apparatus of claim 1 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating-point format having 1 bit for a sign, 4 bits for an exponent, and three bits for a fraction.
7. The apparatus of claim 1 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating-point format having either 1 bit for a sign, 4 bits for an exponent, and three bits for a fraction, or 1 bit for a sign, 5 bits for an exponent, and two bits for a fraction.
8. The apparatus of claim 1 , wherein the opcode is to further indicate the execution circuitry is to zero rows of the source/destination matrix that are not involved in the accumulation.
9. A method comprising:
decoding an instance of a single instruction having one or more fields for an opcode, one or more fields to identify a source/destination matrix operand, one or more fields to identify a first plurality of source matrix operands, one or more fields to identify a second plurality of matrix operands, wherein the opcode is to indicate that execution circuitry is to select a proper subset of 8-bit floating point (FP8) data elements from the first plurality of source matrix operands based on sparsity controls from a first matrix operand of the second plurality of matrix operands, for each element position of the source/destination matrix operand, convert pairs of FP8 elements, as selected, from a row of the first source matrix operands and pairs of FP8 elements from a column of one of the second source matrix operands to 32-bit floating point (FP32), multiply converted even FP8 elements from the identified source matrix operands to generate a first product and separately multiply converted odd FP8 elements from the specified source matrix operands to generate second product, and accumulate the first and second products with previous contents of the source/destination matrix operand, wherein to convert a F8 element at least comprises applying an exponent variable bias, provided by an operand of the insruction, during a conversion; and
executing the decoded single instruction according to the opcode.
10. The method of claim 9 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating point format.
11. The method of claim 10 , wherein the sparsity controls are to select four data elements from the first plurality of source matrix operands per row.
12. The method of claim 9 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating-point format having 1 bit for a sign, 5 bits for an exponent, and two bits for a fraction.
13. The method of claim 12 , wherein the sparsity controls are to select four data elements from the first plurality of source matrix operands per row.
14. The method of claim 12 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating-point format having 1 bit for a sign, 4 bits for an exponent, and three bits for a fraction.
15. The method of claim 12 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating-point format having either 1 bit for a sign, 4 bits for an exponent, and three bits for a fraction, or 1 bit for a sign, 5 bits for an exponent, and two bits for a fraction.
16. The method of claim 9 , wherein the opcode is to further indicate the execution circuitry is to zero rows of the source/destination matrix that are not involved in the accumulation.
17. A non-transitory machine-readable medium that stores program code that when executed by a machine causes the machine to perform a method comprising:
decoding an instance of a single instruction having one or more fields for an opcode, one or more fields to identify a source/destination matrix operand, one or more fields to identify a first plurality of source matrix operands, one or more fields to identify a second plurality of matrix operands, wherein the opcode is to indicate that execution circuitry is to select a proper subset of 8-bit floating point (FP8) data elements from the first plurality of source matrix operands based on sparsity controls from a first matrix operand of the second plurality of matrix operands, for each element position of the source/destination matrix operand, convert pairs of FP8 elements, as selected, from a row of the first source matrix operands and pairs of FP8 elements from a column of one of the second source matrix operands to 32-bit floating point (FP32), multiply converted even FP8 elements from the identified source matrix operands to generate a first product and separately multiply converted odd FP8 elements from the specified source matrix operands to generate second product, and accumulate the first and second products with previous contents of the source/destination matrix operand, wherein to convert a FP8 element at least comprises applying an exponent variable bias, provided by an operand of the instruction, during a conversion; and
executing the decoded single instruction according to the opcode.
18. The non-transitory machine-readable medium of claim 17 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating point format.
19. The non-transitory machine-readable medium of claim 18 , wherein the sparsity controls are to select four data elements from the first plurality of source matrix operands per row.
20. The non-transitory machine-readable medium of claim 17 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating-point format having 1 bit for a sign, 5 bits for an exponent, and two bits for a fraction.
21. The non-transitory machine-readable medium of claim 20 , wherein the sparsity controls are to select four data elements from the first plurality of source matrix operands per row.
22. The non-transitory machine-readable medium of claim 20 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating-point format having 1 bit for a sign, 4 bits for an exponent, and three bits for a fraction.
23. The non-transitory machine-readable medium of claim 20 , wherein elements of the first plurality of source matrix operands are in an 8-bit floating-point format having either 1 bit for a sign, 4 bits for an exponent, and three bits for a fraction, or 1 bit for a sign, 5 bits for an exponent, and two bits for a fraction.