Compute optimizations for low precision machine learning operations
One embodiment provides an accelerator module comprising a memory stack including multiple memory dies; a graphics processing unit (GPU) coupled with the memory stack via one or more memory controllers, the GPU including a plurality of multiprocessors having a single instruction, multiple thread (SIMT) architecture, the multiprocessors to execute at least one single instruction. The at least one single instruction is to cause at least a portion of the GPU to perform a floating point operation on input having differing precisions. The floating point operation is a two-dimensional matrix multiply and accumulate operation.
1. A general-purpose graphics processing unit comprising:
a dynamic precision floating-point unit including a functional unit to perform variable precision floating-point operations, the dynamic precision floating-point unit including:
a hardware control unit having a precision tracker to track an available number of bits of precision for computed data relative to a target precision; and
a numerical transform unit to perform an intermediate numerical transform on computed data when the computed data reaches a threshold precision, wherein the numerical transform unit is to perform a multiplication operation using a logarithm to transform an intermediate value to prevent an underflow during a sequence of operations and perform a division operation using a logarithm to reverse the transform.
2. The general-purpose graphics processing unit as in claim 1 , wherein the dynamic precision floating-point unit includes functional units to perform variable precision floating-point operations, the functional units including native floating-point hardware to perform the variable precision floating-point operations.
3. The general-purpose graphics processing unit as in claim 2 , wherein the numerical transform unit is to perform a division operation using a logarithm to transform an intermediate value to prevent an overflow during a sequence of operations and perform a multiplication operation using a logarithm to reverse transform.
4. The general-purpose graphics processing unit as in claim 3 , wherein the dynamic precision floating-point unit includes a set of registers to store input data and intermediate data at multiple precisions, wherein the set of registers is to store a significand portion of an input data value separately from an exponent portion of the input data value.
5. The general-purpose graphics processing unit as in claim 4 , wherein the set of registers includes an error accumulator to track an accumulated error over a set of floating-point operations.
6. The general-purpose graphics processing unit as in claim 1 , wherein the dynamic precision floating-point unit includes a significand block that includes a dynamic precision adder configurable to add or subtract input data at multiple precisions.
7. The general-purpose graphics processing unit as in claim 6 , the significand block including a dynamic precision multiplier configurable to add or multiply or divide input data at multiple precisions.
8. The general-purpose graphics processing unit as in claim 7 , wherein the dynamic precision floating-point unit includes an exponent block that includes a dynamic precision adder configurable to add or subtract exponents of input data at multiple precisions.
9. The general-purpose graphics processing unit as in claim 8 , the exponent block and the significand block to perform a first floating-point operation to output a first output value having 16-bits of precision.
10. The general-purpose graphics processing unit as in claim 9 , the exponent block and the significand block to perform a second operation to output a second output value having 32-bits of precision.
11. The general-purpose graphics processing unit as in claim 10 , the exponent block and the significand block to perform a third floating-point operation on input data having 32-bit values to output a third output value having a 32-bit data type, the third output value generated at 16-bits of precision.
12. The general-purpose graphics processing unit as in claim 11 , the exponent block including an 8-bit multiplier, and wherein the exponent block and the significand block are configurable to perform a dual 8-bit integer operation.
13. A method comprising:
computing, on a parallel processor device, an activation map based on filter and feature map data associated with a layer of a neural network;
tracking, within a dynamic precision floating-point unit of the parallel processor device, a loss of precision that occurs during computation of the activation map for the layer of the neural network;
determining that intermediate data for computing the activation map has reached a threshold loss in precision; and
transforming the intermediate data of the layer of the neural network via the dynamic precision floating-point unit to reduce error due to precision loss, wherein transforming the intermediate data of the layer of the neural network includes performing a multiplication operation using a logarithm to transform an intermediate value to prevent an underflow during a sequence of operations and performing a division operation using a logarithm to reverse the transform.
14. The method as in claim 13 , additionally comprising:
configuring the dynamic precision floating-point unit to automatically transform intermediate data upon reaching a threshold loss in precision; and
automatically transforming the intermediate data of the layer of the neural network upon reaching the threshold loss in precision.
15. The method as in claim 14 , additionally comprising:
outputting a precision loss status flag via the dynamic precision floating-point unit when intermediate data is within a threshold of precision loss.
16. A data processing system comprising:
a memory stack including multiple memory dies; and
a general-purpose graphics processing unit (GPGPU) coupled with the memory stack via one or more memory controllers, the GPGPU including a plurality of multiprocessors having a single instruction, multiple thread (SIMT) architecture, the GPGPU including a dynamic precision floating-point unit including a functional unit to perform variable precision floating-point operations, the dynamic precision floating-point unit including a hardware control unit having a precision tracker to track an available number of bits of precision for computed data relative to a target precision and a numerical transform unit to perform an intermediate numerical transform on computed data when the computed data reaches a threshold precision, wherein the numerical transform unit is to perform a multiplication operation using a logarithm to transform an intermediate value to prevent an underflow during a sequence of operations and perform a division operation using a logarithm to reverse the transform.
17. The data processing system as in claim 16 , wherein the dynamic precision floating-point unit includes functional units to perform variable precision floating-point operations, the functional units including native floating-point hardware to perform the variable precision floating-point operations.
18. The data processing system as in claim 17 , wherein the numerical transform unit is to perform a division operation using a logarithm to transform an intermediate value to prevent an overflow during a sequence of operations perform a multiplication operation using a logarithm to reverse the transform.
19. The data processing system as in claim 18 , wherein the dynamic precision floating-point unit includes a set of registers to store input data and intermediate data at multiple precisions, wherein the set of registers is to store a significand portion of an input data value separately from an exponent portion of the input data value.
20. The data processing system as in claim 19 , wherein the set of registers includes an error accumulator to track an accumulated error over a set of floating-point operations.