Dynamic precision management for integer deep learning primitives
One embodiment provides for a graphics processing unit to perform computations associated with a neural network, the graphics processing unit comprising compute unit including a hardware logic unit having dynamic precision fixed-point logic, the compute unit to receive a set of dynamic fixed-point tensors, compute, via the dynamic precision fixed-point logic, a right-shift value using an absolute maximum value within the set of dynamic fixed-point tensors and a dynamic range of the set of dynamic fixed-point tensors, right-shift data values within the set of dynamic fixed-point tensors based on the right-shift value, increment a shared exponent associated with the set of dynamic fixed-point tensors based on the right-shift value, perform a compute operation on the set of dynamic fixed-point tensors, and generate an output tensor via the compute operation on the set of dynamic fixed-point tensors.
1. A graphics processing unit to perform computations associated with a neural network, the graphics processing unit comprising:
compute unit including a hardware logic unit having dynamic precision fixed-point logic, the compute unit to:
receive a set of dynamic fixed-point tensors;
compute, via the dynamic precision fixed-point logic, a right-shift value using an absolute maximum value within the set of dynamic fixed-point tensors and a dynamic range of the set of dynamic fixed-point tensors;
right-shift data values within the set of dynamic fixed-point tensors based on the right-shift value;
increment a shared exponent associated with the set of dynamic fixed-point tensors based on the right-shift value;
perform a compute operation on the set of dynamic fixed-point tensors; and
generate an output tensor via the compute operation on the set of dynamic fixed-point tensors.
2. The graphics processing unit as in claim 1 , the dynamic precision fixed-point logic of the compute unit including an integer compute unit including a multiplier, an adder, an accumulator, a shifter, and a register.
3. The graphics processing unit as in claim 2 , wherein register is to store the shared exponent associated with the set of dynamic fixed-point tensors.
4. The graphics processing unit as in claim 1 , the compute unit further to:
determine whether a leading zero count of the absolute maximum value of the output tensor is above a threshold; and
adjust a precision associated with the output tensor in response to determining whether the leading zero count is above the threshold.
5. The graphics processing unit as in claim 4 , wherein to adjust the precision associated with the output tensor includes to adjust a right-shift value within a right-shift counter and adjust a shared exponent for the output tensor.
6. The graphics processing unit as in claim 4 , wherein to adjust the precision associated with the output tensor includes to increase a right-shift value within a right-shift counter and increment a shared exponent in response to determination that the leading zero count is above the threshold.
7. The graphics processing unit as in claim 4 , wherein to adjust the precision associated with the output tensor includes to decrease a right-shift value within a right-shift counter and decrement a shared exponent in response to determination that the leading zero count is below the threshold.
8. The graphics processing unit as in claim 4 , the compute unit to additionally to:
perform an additional compute operation, the additional compute operation having input including the output tensor; and
adjust the precision associated with the output tensor based on the additional compute operation.
9. A method comprising:
receiving a set of dynamic fixed-point tensors;
computing a right-shift value using an absolute maximum value within the set of dynamic fixed-point tensors and a dynamic range of the set of dynamic fixed-point tensors;
right-shifting data values within the set of dynamic fixed-point tensors;
incrementing a shared exponent associated with the set of dynamic fixed-point tensors based on an amount of right-shift applied to the data values within the set of dynamic fixed-point tensors; and
performing a compute operation on the set of dynamic fixed-point tensors.
10. The method as in claim 9 , wherein performing the compute operation on the set of dynamic fixed-point tensors includes:
generating an output tensor via the compute operation on the set of dynamic fixed-point tensors;
determining whether a leading zero count of the absolute maximum value of the output tensor is above a threshold; and
adjusting a precision associated with the output tensor in response to determining whether the leading zero count is above the threshold.
11. The method as in claim 10 , wherein adjusting the precision associated with the output tensor includes adjusting a right-shift counter and shared exponent for the output tensor.
12. The method as in claim 10 , wherein adjusting the precision associated with the output tensor includes increasing a right-shift counter and incrementing a shared exponent in response to determining that the leading zero count is above the threshold.
13. The method as in claim 10 , wherein adjusting the precision associated with the output tensor includes decreasing a right-shift counter and decrementing a shared exponent in response to determining that the leading zero count is below the threshold.
14. The method as in claim 10 , additionally comprising:
performing additional compute operations using the output tensor; and
adjusting the precision associated with the output tensor based on the additional compute operations.
15. A data processing system comprising:
a memory to store a set of dynamic fixed-point tensors; and
one or more processors including at least one graphics processor, the at least one graphics processor including a compute unit including a hardware logic unit having dynamic precision fixed-point logic, the compute unit to:
receive the set of dynamic fixed-point tensors;
compute, via the dynamic precision fixed-point logic, a right-shift value using an absolute maximum value within the set of dynamic fixed-point tensors and a dynamic range of the set of dynamic fixed-point tensors;
right-shift data values within the set of dynamic fixed-point tensors based on the right-shift value;
increment a shared exponent associated with the set of dynamic fixed-point tensors based on the right-shift value;
perform a compute operation on the set of dynamic fixed-point tensors; and
generate an output tensor via the compute operation on the set of dynamic fixed-point tensors.
16. The data processing system as in claim 15 , the compute unit further to:
determine whether a leading zero count of the absolute maximum value of the output tensor is above a threshold; and
adjust a precision associated with the output tensor in response to determining whether the leading zero count is above the threshold.
17. The data processing system as in claim 16 , wherein to adjust the precision associated with the output tensor includes to adjust a right-shift value within a right-shift counter and adjust a shared exponent for the output tensor.
18. The data processing system as in claim 16 , wherein to adjust the precision associated with the output tensor includes to increase a right-shift value within a right-shift counter and increment a shared exponent in response to determination that the leading zero count is above the threshold.
19. The data processing system as in claim 16 , wherein to adjust the precision associated with the output tensor includes to decrease a right-shift value within a right-shift counter and decrement a shared exponent in response to determination that the leading zero count is below the threshold.
20. The data processing system as in claim 16 , the compute unit to additionally to:
perform an additional compute operation, the additional compute operation having input including the output tensor; and
adjust the precision associated with the output tensor based on the additional compute operation.