Instructions and logic to perform floating point and integer operations for machine learning
One embodiment provides a graphics processor comprising a memory controller and a graphics processing resource coupled with the memory controller. The graphics processing resource includes circuitry configured to execute an instruction to perform a matrix operation on first input including weight data and second input including input activation data, generate intermediate data based on a result of the matrix operation, quantize the intermediate data to a floating-point format determined based on a statistical distribution of first output data, and output, as second output data, quantized intermediate data in a determined floating-point format.
1. A graphics processor comprising:
a memory controller; and
a graphics multiprocessor coupled with the memory controller, the graphics multiprocessor including first circuitry configured to:
execute an instruction to perform a matrix operation on first input and second input;
generate intermediate data based on a result of the matrix operation;
convert the intermediate data to a floating-point format determined based on statistics associated with first output data; and
output, as second output data, converted intermediate data in a determined floating-point format.
2. The graphics processor as in claim 1 , wherein the matrix operation includes a matrix multiply operation.
3. The graphics processor as in claim 1 , wherein the first circuitry is configured to generate the intermediate data in one of a first plurality of floating-point formats.
4. The graphics processor as in claim 3 , wherein the first plurality of floating-point formats includes a 16-bit format and a 32-bit format.
5. The graphics processor as in claim 1 , wherein the first circuitry is configured to convert the intermediate data to one of a second plurality of floating-point formats.
6. The graphics processor as in claim 5 , wherein the second plurality of floating-point formats includes multiple floating-point formats having a same number of bits and a different number of exponent bits.
7. The graphics processor as in claim 6 , wherein the multiple floating-point formats have a different number of mantissa bits.
8. The graphics processor as in claim 1 , wherein the first output data includes output generated for a first layer of a neural network.
9. The graphics processor as in claim 8 , wherein the second output data includes output generated for a second layer of a neural network.
10. The graphics processor as in claim 9 , wherein the first output data includes data generated by the first circuitry prior to generation of the intermediate data.
11. The graphics processor as in claim 1 , wherein the statistics associated with the first output data includes a dynamic range of the first output data.
12. The graphics processor as in claim 1 , further comprising second circuitry to determine statistics associated with first output data.
13. The graphics processor as in claim 12 , wherein the memory controller includes the second circuitry.
14. The graphics processor as in claim 13 , wherein the memory controller is configured to determine, via the second circuitry, the statistics associated with first output data during a write of the first output data to memory.
15. The graphics processor as in claim 14 , wherein the first circuitry is configured to request a write of the second output data to memory via the memory controller.
16. The graphics processor as in claim 15 , wherein the memory controller is configured to determine, via the second circuitry, statistics associated with the second output data during the write of the second output data to memory.
17. A method comprising:
executing an instruction on a graphics processor to perform a matrix operation on first input and second input;
generating intermediate data based on a result of the matrix operation;
converting the intermediate data to a floating-point format determined based on statistics associated with first output data; and
outputting, as second output data, converted intermediate data in a determined floating-point format.
18. The method as in claim 17 , further comprising:
generating the intermediate data in one of a first plurality of floating-point formats, wherein the first plurality of floating-point formats includes a 16-bit format and a 32-bit format; and
converting the intermediate data to one of a second plurality of floating-point formats, wherein the second plurality of floating-point formats includes multiple floating-point formats having a same number of bits and a different number of exponent bits.
19. The method as in claim 18 , wherein the multiple floating-point formats have a different number of mantissa bits.
20. A data processing system comprising:
a memory device to store an instruction; and
a graphics processor coupled with the memory device and configured to execute the instruction, the graphics processor including a memory controller and a graphics multiprocessor coupled with the memory controller, the graphics processor including circuitry configured to:
execute an instruction to perform a matrix operation on first input and second input;
generate intermediate data based on a result of the matrix operation;
convert the intermediate data to a floating-point format determined based on statistics associated with first output data; and
output, as second output data, converted intermediate data in a determined floating-point format.
21. The data processing system as in claim 20 , wherein the matrix operation includes a matrix multiply operation, the circuitry is configured to generate the intermediate data in one of a first plurality of floating-point formats, and the first plurality of floating-point formats includes a 16-bit format and a 32-bit format.
22. The data processing system as in claim 20 , wherein the circuitry is configured to convert the intermediate data to one of a second plurality of floating-point formats, the second plurality of floating-point formats including multiple floating-point formats having a same number of bits and a different number of exponent bits.
23. The data processing system as in claim 22 , wherein the multiple floating-point formats have a different number of mantissa bits.
24. The data processing system as in claim 20 , the graphics processor including circuitry configured to determine statistics associated with first output data.
25. The data processing system as in claim 24 , wherein the circuitry configured to determine statistics associated with first output data is included in the memory controller of the graphics processor.