Instructions and logic to perform floating point and integer operations for machine learning
A processing apparatus is provided comprising a multiprocessor having a multithreaded architecture. The multiprocessor can execute at least one single instruction to perform parallel mixed precision matrix operations. In one embodiment the apparatus includes a memory interface and an array of multiprocessors coupled to the memory interface. At least one multiprocessor in the array of multiprocessors is configured to execute a fused multiply-add instruction in parallel across multiple threads.
1. An apparatus comprising:
a memory interface;
an array of multiprocessors coupled to the memory interface, at least one multiprocessor in the array of multiprocessors to execute a fused multiply-add instruction in parallel across multiple threads, the at least one multiprocessor comprising:
a register file to store data; and
execution circuitry coupled to the register file, the execution circuitry to execute the fused multiply-add instruction to generate a multidimensional result matrix, the execution circuitry comprising:
hardware logic to convert a first plurality of data elements of a first multidimensional source matrix and a second plurality of data elements of a second multidimensional source matrix from a 32-bit floating point data format to a reduced precision floating point format having a 1-bit sign, an 8-bit exponent, and a mantissa, the mantissa of the reduced precision floating point format having fewer bits than a mantissa of the 32-bit floating point data format; and
a plurality of multiply-add circuits to perform parallel fused multiply-add operations to multiply the first plurality of data elements in the reduced precision floating point format by corresponding data elements of the second plurality of data elements in the reduced precision floating point format to generate a plurality of products, and to add the plurality of products to corresponding 32-bit floating point values to generate corresponding 32-bit_floating point data elements of the multidimensional result matrix.
2. The apparatus of claim 1 wherein the mantissa of the reduced precision floating point format comprises a 7-bit mantissa.
3. The apparatus of claim 1 wherein the fused multiply-add instruction comprises a first operand to identify the first plurality of data elements and a second operand to identify the second plurality of data elements.
4. The apparatus of claim 3 wherein the first operand identifies the first plurality of data elements in a first one or more registers of the register file and the second operand identifies the second plurality of data elements in a second one or more registers of the register file.
5. The apparatus of claim 1 , further comprising:
an instruction fetch unit to fetch the fused multiply-add instruction;
a decoder to decode the fused multiply-add instruction to generate parallel multiply-add operations to be performed across the multiple threads; and
a scheduler to schedule the parallel multiply-add operations for execution by the execution circuitry.
6. The apparatus of claim 5 wherein the execution circuitry comprises a plurality of arithmetic logic units (ALUs).
7. The apparatus of claim 1 further comprising:
an interconnect fabric to couple the array of multiprocessors to the memory interface.
8. The apparatus of claim 7 further comprising:
an input/output (IO) interface coupled to the interconnect fabric.
9. The apparatus of claim 8 further comprising:
a shared cache or local memory shared by the plurality of multiply-add circuits.
10. A method comprising:
communicatively coupling an array of multiprocessors to a memory interface, at least one multiprocessor in the array of multiprocessors to execute a fused multiply-add instruction in parallel across multiple threads, the at least one multiprocessor comprising a register file to store data and execution circuitry coupled to the register file;
executing the fused multiply-add instruction by the execution circuitry to generate a multidimensional result matrix, wherein executing comprises:
converting a first plurality of data elements of a first multidimensional source matrix and a second plurality of data elements of a second multidimensional source matrix from a 32-bit floating point data format to a reduced precision floating point format having a 1-bit sign, an 8-bit exponent, and a mantissa, the mantissa of the reduced precision floating point format having fewer bits than a mantissa of the 32-bit floating point data format; and
performing parallel fused multiply-add operations on a plurality of multiply-add circuits to multiply the first plurality of data elements in the reduced precision floating point format by corresponding data elements of the second plurality of data elements in the reduced precision floating point format to generate a plurality of products, and to add the plurality of products to corresponding 32-bit floating point values to generate corresponding 32-bit_floating point data elements of the multidimensional result matrix.
11. The method of claim 10 wherein the mantissa of the reduced precision floating point format comprises a 7-bit mantissa.
12. The method of claim 10 wherein the fused multiply-add instruction comprises a first operand to identify the first plurality of data elements and a second operand to identify the second plurality of data elements.
13. The method of claim 12 wherein the first operand identifies the first plurality of data elements in a first one or more registers of the register file and the second operand identifies the second plurality of data elements in a second one or more registers of the register file.
14. The method claim 10 , further comprising:
fetching the fused multiply-add instruction;
decoding the fused multiply-add instruction to generate parallel multiply-add operations to be performed across the multiple threads; and
scheduling the parallel multiply-add operations for execution by the execution circuitry.
15. The method of claim 14 wherein the execution circuitry comprises a plurality of arithmetic logic units (ALUs).
16. The method of claim 10 further comprising:
communicatively coupling the array of multiprocessors to the memory interface over an interconnect fabric; and
communicatively coupling an input/output (IO) interface to the interconnect fabric.
17. The method of claim 16 further comprising:
sharing a cache or local memory between the plurality of multiply-add circuits.
18. A system comprising:
a system memory;
a memory interface coupled to the system memory;
an array of multiprocessors coupled to the memory interface, at least one multiprocessor in the array of multiprocessors to execute a fused multiply-add instruction in parallel across multiple threads, the at least one multiprocessor comprising:
a register file to store data; and
execution circuitry coupled to the register file, the execution circuitry to execute the fused multiply-add instruction to generate a multidimensional result matrix, the execution circuitry comprising:
hardware logic to convert a first plurality of data elements of a first multidimensional source matrix and a second plurality of data elements of a second multidimensional source matrix from a 32-bit floating point data format to a reduced precision floating point format having a 1-bit sign, an 8-bit exponent, and a mantissa, the mantissa of the reduced precision floating point format having fewer bits than a mantissa of the 32-bit floating point data format; and
a plurality of multiply-add circuits to perform parallel fused multiply-add operations to multiply the first plurality of data elements in the reduced precision floating point format by corresponding data elements of the second plurality of data elements in the reduced precision floating point format to generate a plurality of products, and to add the plurality of products to corresponding 32-bit floating point values to generate corresponding 32-bit_floating point data elements of the multidimensional result matrix.
19. The system of claim 18 wherein the mantissa of the reduced precision floating point format comprises a 7-bit mantissa.
20. The system of claim 18 wherein the fused multiply-add instruction comprises a first operand to identify the first plurality of data elements and a second operand to identify the second plurality of data elements.
21. The system of claim 20 wherein the first operand identifies the first plurality of data elements in a first one or more registers of the register file and the second operand identifies the second plurality of data elements in a second one or more registers of the register file.
22. The system claim 18 , further comprising:
an instruction fetch unit to fetch the fused multiply-add instruction;
a decoder to decode the fused multiply-add instruction to generate parallel multiply-add operations to be performed across the multiple threads; and
a scheduler to schedule the parallel multiply-add operations for execution by the execution circuitry.
23. The system of claim 22 wherein the execution circuitry comprises a plurality of arithmetic logic units (ALUs).
24. The system of claim 19 further comprising:
an interconnect fabric to couple the array of multiprocessors to the memory interface; and
an input/output (IO) interface coupled to the interconnect fabric.
25. The system of claim 24 further comprising:
a shared cache or local memory shared by the plurality of multiply-add circuits.
26. The system of claim 18 further comprising:
a local memory;
a local memory interface coupled to the local memory; and
a memory controller to route memory requests to the local memory via the local memory interface or to the system memory via the memory interface.
27. The system of claim 24 further comprising:
an I/O device coupled to the I/O interface.
28. The system of claim 27 wherein the I/O device comprises one or more of:
a persistent storage device; a network interface device; and a serial communication device.
29. The system of claim 28 further comprising:
a host processor coupled to the I/O interface via a host processor interface, the host processor to execute program code and provide commands to be executed by the execution circuitry.
30. An apparatus comprising:
a memory interface;
an array of multiprocessors coupled to the memory interface, at least one multiprocessor in the array of multiprocessors to execute a fused multiply-add instruction in parallel across multiple threads, the at least one multiprocessor comprising:
a register file to store data; and
execution circuitry coupled to the register file, the execution circuitry to execute the fused multiply-add instruction to generate a multidimensional result matrix, the execution circuitry comprising:
a local shared memory to store a first plurality of data elements of a first multidimensional source matrix and a second plurality of data elements of a second multidimensional source matrix in a reduced precision floating point format comprising a 1-bit sign, a 7-bit mantissa and an 8-bit exponent; and
a plurality of multiply-add circuits to perform parallel fused multiply-add operations to multiply the first plurality of data elements in the reduced precision floating point format by corresponding data elements of the second plurality of data elements in the reduced precision floating point format to generate a plurality of products, and to add the plurality of products to corresponding 32-bit floating point values to generate corresponding 32-bit floating point data elements of the multidimensional result matrix.