Dynamic precision management for integer deep learning primitives
One embodiment provides for a graphics processing unit to perform computations associated with a neural network, the graphics processing unit comprising a hardware processing unit having a dynamic precision fixed-point unit that is configurable to convert elements of a floating-point tensor to convert the floating-point tensor into a fixed-point tensor.
1. A graphics processing unit comprising:
an interface coupled to an interconnect fabric; and
a graphics core cluster including a plurality of multiprocessors that are interconnected via a data interconnect, a multiprocessor of the plurality of multiprocessors including circuitry configured to:
receive an input tensor associated with an operation to be performed via the graphics core cluster, the input tensor including data elements in a floating-point format;
divide the data elements of the input tensor into multiple blocks of data elements;
determine a shared exponent for each of the multiple blocks of data elements;
generate metadata for the multiple blocks to indicate the shared exponent for each of the multiple blocks of data elements;
perform a compute operation on input including at least one of the multiple blocks of data elements and the metadata; and
generate an output tensor via the compute operation.
2. The graphics processing unit of claim 1 , the circuitry to convert the data elements of the at least one of the multiple blocks of data elements from the floating-point format to a fixed-point format having the shared exponent, the input including the at least one of the multiple blocks of data elements in the fixed-point format.
3. The graphics processing unit of claim 2 , wherein to convert the data elements, the circuitry is to be configured to compute a right-shift value based on the shared exponent and right-shift a mantissa of the element based on the right-shift value to generate a magnitude integer.
4. The graphics processing unit of claim 3 , the circuitry to determine the shared exponent based on a scale factor associated with the input tensor.
5. The graphics processing unit of claim 4 , wherein the scale factor is a run-time configurable scale factor.
6. The graphics processing unit of claim 5 , wherein the fixed-point format is a dynamic precision fixed-point format and the circuitry includes an integer compute unit having dynamic precision fixed-point circuitry.
7. The graphics processing unit of claim 6 , wherein the integer compute unit includes a multiplier, an adder, and accumulator, a shifter, and a register.
8. The graphics processing unit of claim 7 , wherein the register is to store the run-time configurable scale factor.
9. The graphics processing unit of claim 8 , wherein the dynamic precision fixed-point circuitry is configured to adjust the run-time configurable scale factor based on output from the multiplier.
10. A method comprising:
receiving an input tensor associated with an operation to be performed via a graphics core cluster including a plurality of multiprocessors that are interconnected via a data interconnect, the input tensor including data elements in a floating-point format;
dividing the data elements of the input tensor into multiple blocks of data elements;
determining a shared exponent for each of the multiple blocks of data elements;
generating metadata for the multiple blocks to indicate the shared exponent for each of the multiple blocks of data elements;
performing a compute operation on input including at least one of the multiple blocks of data elements and the metadata; and
generating an output tensor via the compute operation.
11. The method of claim 10 , comprising converting the data elements of the at least one of the multiple blocks of data elements from the floating-point format to a fixed-point format having the shared exponent, the input including the at least one of the multiple blocks of data elements in the fixed-point format.
12. The method of claim 11 , wherein converting the data elements includes computing a right-shift value based on the shared exponent and right-shift a mantissa of the element based on the right-shift value to generate a magnitude integer.
13. The method of claim 12 , comprising determining the shared exponent based on a scale factor associated with the input tensor.
14. The method of claim 13 , wherein the scale factor is a run-time configurable scale factor.
15. The method of claim 14 , wherein the fixed-point format is a dynamic precision fixed-point format.
16. A graphics processing system comprising:
a memory device; and
a graphics core cluster coupled with the memory device, the graphics core cluster including a plurality of multiprocessors that are interconnected via a data interconnect, a multiprocessor of the plurality of multiprocessors including circuitry configured to:
receive an input tensor associated with an operation to be performed via the graphics core cluster, the input tensor including data elements in a floating-point format;
divide the data elements of the input tensor into multiple blocks of data elements;
determine a shared exponent for each of the multiple blocks of data elements;
generate metadata for the multiple blocks to indicate the shared exponent for each of the multiple blocks of data elements;
perform a compute operation on input including at least one of the multiple blocks of data elements and the metadata; and
generate an output tensor via the compute operation.
17. The graphics processing system of claim 16 , the circuitry to convert the data elements of the at least one of the multiple blocks of data elements from the floating-point format to a fixed-point format having the shared exponent, the input including the at least one of the multiple blocks of data elements in the fixed-point format.
18. The graphics processing system of claim 17 , wherein to convert the data elements, the circuitry is to be configured to compute a right-shift value based on the shared exponent and right-shift a mantissa of a data element based on the right-shift value to generate a magnitude integer.
19. The graphics processing system of claim 18 , the circuitry to determine the shared exponent based on a scale factor associated with the input tensor, the scale factor a run-time configurable scale factor.
20. The graphics processing system of claim 19 , wherein the fixed-point format is a dynamic precision fixed-point format and the circuitry includes an integer compute unit having dynamic precision fixed-point circuitry.
21. A non-transitory machine-readable medium having code stored thereon which, when executed by a machine, causes the machine to perform operations, comprising:
receiving an input tensor associated with an operation to be performed by a processing circuit comprising one or more field programmable gate arrays (FPGAs), programmable processors, or a graphics core cluster, the graphics core cluster including a plurality of multiprocessors that are interconnected via a data interconnect, the input tensor including data elements in a floating-point format;
dividing the data elements of the input tensor into multiple blocks of data elements;
determining a plurality of exponents, each exponent of the plurality of exponents shared by data elements of a corresponding block of data elements;
generating metadata for the multiple blocks of data elements, the metadata indicating the plurality of exponents;
performing compute operations based on the input tensor, including a compute operation based on at least one of the multiple blocks of data elements and the metadata; and
generating an output tensor from the compute operation.
22. The non-transitory machine-readable medium of claim 21 , comprising converting the data elements of the at least one of the multiple blocks of data elements from the floating-point format to a fixed-point format having the shared exponent, the input including the at least one of the multiple blocks of data elements in the fixed-point format.
23. The non-transitory machine-readable medium of claim 22 , wherein converting the data elements includes computing a right-shift value based on the shared exponent and right-shift a mantissa of the element based on the right-shift value to generate a magnitude integer.
24. The non-transitory machine-readable medium of claim 23 , comprising determining the shared exponent based on a scale factor associated with the input tensor.
25. The non-transitory machine-readable medium of claim 24 , wherein the scale factor is a run-time configurable scale factor.
26. The non-transitory machine-readable medium of claim 25 , wherein the fixed-point format is a dynamic precision fixed-point format.