Dynamic precision management for integer deep learning primitives

1. A graphics processing unit to perform computations associated with a neural network, the graphics processing unit comprising:

a compute unit including a hardware logic unit having dynamic precision fixed-point logic, the compute unit to:

receive a floating-point tensor;

quantize elements of the floating-point tensor to convert the floating-point tensor into a dynamic fixed-point tensor;

perform a compute operation on at least the dynamic fixed-point tensor;

generate an output tensor via the compute operation;

determine whether a leading zero count of an absolute maximum value of the output tensor is above a threshold; and

adjust a precision associated with the output tensor in response to determining whether the leading zero count is above the threshold.

2. The graphics processing unit as in claim 1 , the dynamic precision fixed-point logic of the compute unit including an integer compute unit including a multiplier, an adder, an accumulator, a shifter, and a register.

3. The graphics processing unit as in claim 2 , wherein the register is to store a shared exponent associated with the dynamic fixed-point tensor, and the shared exponent is shared by elements of the dynamic fixed-point tensor.

4. The graphics processing unit as in claim 3 , wherein to quantize an element of the floating-point tensor, the compute unit is to:

compute, via the dynamic precision fixed-point logic, a right-shift value based on a difference between an exponent value of the element of the floating-point tensor and the exponent value of an absolute maximum value of the floating-point tensor; and

right-shift a mantissa of the element based on the right-shift value to generate a magnitude integer.

5. The graphics processing unit as in claim 4 , wherein an implicit leading bit of the mantissa is stored as an explicit bit within the magnitude integer.

6. The graphics processing unit as in claim 4 , the compute unit to additionally to:

round the magnitude integer according to value of a first bit and a second bit within the magnitude integer to generate a rounded magnitude integer.

7. The graphics processing unit as in claim 1 ,

wherein the compute operation is a matrix arithmetic operation.

8. The graphics processing unit as in claim 1 , wherein to adjust the precision associated with the output tensor includes to adjust a right-shift value within a right-shift counter and adjust a shared exponent for the output tensor.

9. The graphics processing unit as in claim 1 , wherein to adjust the precision associated with the output tensor includes to increase a right-shift value within a right-shift counter and increment a shared exponent in response to determination that the leading zero count is above the threshold.

10. The graphics processing unit as in claim 1 , wherein to adjust the precision associated with the output tensor includes to decrease a right-shift value within a right-shift counter and decrement a shared exponent in response to determination that the leading zero count is below the threshold.

11. A method comprising:

receiving a floating-point tensor;

quantizing elements of the floating-point tensor to convert the floating-point tensor into a dynamic fixed-point tensor;

performing a compute operation on at least the dynamic fixed-point tensor;

generating an output tensor via the compute operation;

determining whether a leading zero count of an absolute maximum value of the output tensor is above a threshold; and

adjusting a precision associated with the output tensor in response to determining whether the leading zero count is above the threshold.

12. The method as in claim 11 , wherein quantizing an element of the floating-point tensor includes:

computing a right-shift value based on a difference between an exponent value of an element of the floating-point tensor and the exponent value of an absolute maximum value of the floating-point tensor;

right-shifting a mantissa of the element based on the right-shift value to generate a magnitude integer; and

storing an implicit leading bit of the mantissa as an explicit bit within the magnitude integer.

13. The method as in claim 12 , further comprising rounding the magnitude integer according to value of a first bit and a second bit within the magnitude integer to generate a rounded magnitude integer.

14. The method as in claim 11 ,

wherein the compute operation is a matrix arithmetic operation.

15. A data processing system comprising:

a memory to store a set of dynamic fixed-point tensors; and

one or more processors including at least one graphics processor, the at least one graphics processor including a compute unit including a hardware logic unit having dynamic precision fixed-point logic, the compute unit to:

receive a floating-point tensor;

quantize elements of the floating-point tensor to convert the floating-point tensor into a dynamic fixed-point tensor;

perform a compute operation on at least the dynamic fixed-point tensor;

generate an output tensor via the compute operation;

determine whether a leading zero count of an absolute maximum value of the output tensor is above a threshold; and

adjust a precision associated with the output tensor in response to determining whether the leading zero count is above the threshold.

16. The data processing system as in claim 15 , wherein the dynamic precision fixed-point logic of the compute unit includes an integer compute unit, the integer compute unit including a multiplier, an adder, an accumulator, a shifter, and a register, wherein the register is to store a shared exponent associated with the dynamic fixed-point tensor, and the shared exponent is shared by elements of the dynamic fixed-point tensor.

17. The data processing system as in claim 16 , wherein to quantize an element of the floating-point tensor, the compute unit is to:

compute, via the dynamic precision fixed-point logic, a right-shift value based on a difference between an exponent value of the element of the floating-point tensor and the exponent value of an absolute maximum value of the floating-point tensor;

right-shift a mantissa of the element based on the right-shift value to generate a magnitude integer; and

store an implicit leading bit of the mantissa as an explicit bit within the magnitude integer.

18. The data processing system as in claim 17 , the compute unit to additionally to:

round the magnitude integer according to value of a first bit and a second bit within the magnitude integer to generate a rounded magnitude integer.

19. The data processing system as in claim 15 ,

wherein the compute operation is a matrix arithmetic operation.

20. The data processing system as in claim 19 , wherein to adjust the precision associated with the output tensor includes to adjust a right-shift value within a right-shift counter and adjust a shared exponent for the output tensor.