Integrated circuits with machine learning extensions
An integrated circuit with specialized processing blocks is provided. A specialized processing block may be optimized for machine learning algorithms and may include a multiplier data path that feeds an adder data path. The multiplier data path may be decomposed into multiple partial product generators, multiple compressors, and multiple carry-propagate adders of a first precision. Results from the carry-propagate adders may be added using a floating-point adder of the first precision. Results from the floating-point adder may be optionally cast to a second precision that is higher or more accurate than the first precision. The adder data path may include an adder of the second precision that combines the results from the floating-point adder with zero, with a general-purpose input, or with other dot product terms. Operated in this way, the specialized processing block provides a technical improvement of greatly increasing the functional density for implementing machine learning algorithms.
1. An integrated circuit device comprising:
pre-processing circuitry configurable to receive inputs of a plurality of floating point formats;
a multiplier configurable to output a product, wherein the multiplier is operable on outputs of the pre-processing circuitry;
a multiplexer configurable to:
receive a plurality of inputs, wherein the plurality of inputs comprises a first multiplexer input, a second multiplexer input, and a third multiplexer input, wherein the first multiplexer input is received as an input to a digital signal processing circuit comprising the pre-processing circuitry, the second multiplexer input is received from a first adjacent digital signal processing circuit, and the third multiplexer input comprises the product of the multiplier; and
output one of the plurality of inputs to the multiplexer as a multiplexer output;
an adder configurable to:
receive the multiplexer output and receive a feedback input, wherein the feedback input is based at least in part upon feedback from an adder output of the adder;
add the multiplexer output to the feedback input to generate the adder output; and
output the adder output.
2. The integrated circuit device of claim 1 , wherein the integrated circuit device comprises programmable logic comprising digital signal processing circuitry.
3. The integrated circuit device of claim 2 , wherein the digital signal processing circuitry comprises the pre-processing circuitry, the multiplier, the multiplexer, and the adder.
4. The integrated circuit device of claim 2 , wherein the digital signal processing circuitry is configurable to route the product to a second adjacent digital signal processing circuit.
5. The integrated circuit device of claim 1 , wherein the adder comprises accumulator circuitry.
6. The integrated circuit device of claim 1 , wherein the first multiplexer input is in a first floating point format, the second multiplexer input is in a second floating point format different from the first floating point format, and the third multiplexer input is in a third floating point format different from the first floating point format and the second floating point format.
7. The integrated circuit device of claim 6 , wherein the adder is implemented using an intermediate format different than the first floating point format, the second floating point format, and the third floating point format.
8. The integrated circuit device of claim 1 , wherein the plurality of floating point formats comprises FP16, FP32, or any combination thereof.
9. The integrated circuit device of claim 1 , wherein the plurality of floating point formats comprise BFLOAT16, BFLOAT17, BFLOAT18, BFLOAT19, or any combination thereof.
10. Digital signal processing circuitry, comprising:
pre-processing circuitry configurable to receive one or more inputs in a first mode or a second mode;
a multiplier configurable to output a product, wherein the multiplier is operable on outputs of the pre-processing circuitry;
a multiplexer configurable to:
receive a plurality of inputs, wherein the plurality of inputs comprises a first multiplexer input, a second multiplexer input, and a third multiplexer input, wherein the first multiplexer input is received as an input to the digital signal processing circuit, the second multiplexer input is received from a first adjacent digital signal processing circuit, and the third multiplexer input comprises the product of the multiplier; and
output one of the plurality of inputs to the multiplexer as a multiplexer output; and an accumulator configurable to:
receive the multiplexer output and receive a feedback input, wherein the feedback input is based at least in part upon feedback from an accumulator output of the accumulator;
add the multiplexer output to the feedback input to generate the accumulator output; and
output the accumulator output.
11. The digital signal processing circuitry of claim 10 , wherein the multiplier is operable to receive FP16 or FP32 inputs from the pre-processing circuitry in the first mode.
12. The digital signal processing circuitry of claim 10 , wherein the multiplier is operable to receive BFLOAT16, BFLOAT17, BFLOAT18, BFLOAT19, or any combination thereof from the pre-processing circuitry in the second mode.
13. The digital signal processing circuitry of claim 10 , wherein the digital signal processing circuitry is configurable to route the product to a second adjacent digital signal processing circuit.
14. The digital signal processing circuitry of claim 10 , wherein the first multiplexer input is of a first floating point format, the second multiplexer input is of a second floating point format different from the first floating point format, and the third multiplexer input is of a third floating point format different from the first floating point format and the second floating point format.
15. The digital signal processing circuitry of claim 14 , wherein the accumulator is implemented using an intermediate format different than the first floating point format, the second floating point format, and the third floating point format.
16. A method, comprising:
receiving, at input terminals of digital processing circuitry, a plurality of inputs of a plurality of floating point formats;
multiplying, using a multiplier of the digital processing circuitry, the plurality of inputs to produce a product;
receiving, at a multiplexer of the digital processing circuitry, a first input, wherein the first input is received as an input to a digital signal processing circuit;
receiving, at the multiplexer, a second input, wherein the second input is received from a first adjacent digital signal processing circuit;
receiving, at the multiplexer, a third input, wherein the third input is based at least in part on the product;
outputting, from the multiplexer, one of the first input, the second input, or the third input;
receiving, at an adder of the digital processing circuitry, the one of the first input, the second input, or the third input;
receiving, at the adder, a feedback input, wherein the feedback input is based at least in part on an output;
adding, using the adder, the one of the first input, the second input, or the third input and the feedback input to generate the output; and
outputting the output from the digital processing circuitry.
17. The method of claim 16 , routing the product is to a second adjacent digital signal processing circuit.
18. The method of claim 16 , wherein the plurality of floating point formats comprises FP16, FP32, or any combination thereof.
19. The method of claim 16 , wherein the plurality of floating point formats comprises BFLOAT16, BFLOAT17, BFLOAT18, BFLOAT19, or any combination thereof.
20. The method of claim 16 , wherein the first input is in a first floating point format, the second input is in a second floating point format different from the first floating point format, and the third input is in a third floating point format different from the first floating point format and the second floating point format.