Circuitry for high-bandwidth, low-latency machine learning
The present disclosure relates generally to techniques for efficiently performing operations associated with artificial intelligence (AI), machine learning (ML), and/or deep learning (DL) applications, such as training and/or interference calculations, using an integrated circuit device. More specifically, the present disclosure relates to an integrated circuit design implemented to perform these operations with low latency and/or a high bandwidth of data. For example, embodiments of a computationally dense digital signal processing (DSP) circuitry, implemented to efficiently perform one or more arithmetic operations (e.g., a dot-product) on an input are disclosed. Moreover, embodiments described herein may relate to layout, design, and data scheduling of a processing element array implemented to compute matrix multiplications (e.g., systolic array multiplication).
1. Circuitry implemented on an integrated circuit device, comprising:
input circuitry configurable to receive a first vector of a first matrix, a second vector of the first matrix, and a third vector of a second matrix;
a register communicatively coupled to the input circuitry, wherein the register is configurable to store at least a portion of the second vector;
multiplier circuitry comprising:
a first multiplier configurable to:
receive the first vector directly from the input circuitry and receive the second vector from the register; and
determine a first partial inner-product based at least in part on the first vector and the second vector; and
a second multiplier configurable to determine a second partial inner-product based at least in part on the second vector and the third vector; and
adder circuitry configurable to determine a first inner-product of a dot-product based at least in part on the first partial inner-product.
2. The circuitry of claim 1 , wherein the adder circuitry is configurable to determine a second inner-product of the dot-product based at least in part on the second partial inner-product.
3. The circuitry of claim 1 , wherein the circuitry is implemented in a digital signal processing (DSP) block included on the integrated circuit device.
4. The circuitry of claim 3 , wherein the integrated circuit device comprises programmable logic.
5. The circuitry of claim 1 , comprising selection circuitry communicatively coupled to the input circuitry and the register, wherein the circuitry is configurable to selectively perform a first operation or a second operation based at least in part on an output of the selection circuitry.
6. The circuitry of claim 1 , wherein the second matrix is a vector.
7. The circuitry of claim 1 , wherein the adder circuitry comprises a fused adder.
8. The circuitry of claim 1 , wherein:
the input circuitry is communicatively coupled to output circuitry of additional circuitry configurable to output, at the output circuitry, a third partial inner-product based at least in part on the first vector and the third vector; and
the adder circuitry is configurable to determine the first inner-product based at least in part on the first partial inner-product and the third partial inner-product.
9. The circuitry of claim 1 , wherein:
the first vector comprises a row vector and the second vector comprises a column vector; or
the first vector comprises a column vector and the second vector comprises a row vector.
10. The circuitry of claim 1 , comprising:
a second register communicatively coupled to the input circuitry and configurable to store the at least a portion of the second vector; and
a crossbar disposed between and communicatively coupled to the second register and the multiplier circuitry, wherein the crossbar is configurable to selectively output, to the multiplier circuitry, first data associated with the first vector, second data associated with the second vector, or both.
11. An integrated circuit device comprising:
a cascade chain implemented using a plurality of digital signal processing (DSP) blocks, wherein the cascade chain is configurable to determine a dot-product based at least in part on first data associated with a first matrix and second data associated with a second matrix, wherein the first data comprises a first vector and a second vector of the first matrix, wherein the second data comprises a third vector of the second matrix, wherein a first DSP block of the plurality of DSP blocks comprises:
input circuitry configurable to receive a first vector of a first matrix, a second vector of the first matrix, and a third vector of a second matrix;
a register communicatively coupled to the input circuitry, wherein the register is configurable to store at least a portion of the second vector;
multiplier circuitry comprising:
a first multiplier configurable to:
receive the first vector directly from the input circuitry and receive the second vector from the register; and
determine a first partial inner-product based at least in part on the first vector and the second vector; and
a second multiplier configurable to determine a second partial inner-product based at least in part on the second vector and the third vector; and
adder circuitry configurable to determine a first inner-product of a dot-product based at least in part on the first partial inner-product; and
an embedded memory communicatively coupled to an output of the cascade chain and configurable to:
receive the dot-product from the cascade chain; and
store the dot-product.
12. The integrated circuit device of claim 11 , wherein the plurality of DSP blocks comprises four DSP blocks.
13. The integrated circuit device of claim 11 , wherein the adder circuitry comprises floating-point compressor circuitry.
14. The integrated circuit device of claim 11 , comprising selection circuitry communicatively coupled to an input of the cascade chain, wherein the selection circuitry is configurable to serialize the first data, the second data, or a combination thereof input to the cascade chain.
15. The integrated circuit device of claim 11 , wherein the embedded memory is configurable to implement a first-in, first-out (FIFO) structure.
16. The integrated circuit device of claim 11 , wherein the second data comprises the third vector and a fourth vector of the second matrix.
17. The integrated circuit device of claim 16 , wherein the cascade chain is configurable to:
determine the dot-product based at least in part on the first vector and the third vector;
output the dot-product at first output circuitry;
determine an additional dot-product based at least in part on the first vector and the fourth vector; and
output the additional dot-product at second output circuitry.
18. A system comprising:
a first integrated circuit device comprising:
input circuitry configurable to receive a first vector of a first matrix, a second vector of the first matrix, and a third vector of a second matrix;
a register communicatively coupled to the input circuitry, wherein the register is configurable to store at least a portion of the second vector;
multiplier circuitry comprising:
a first multiplier configurable to:
receive the first vector directly from the input circuitry and receive the second vector from the register; and
determine a first partial inner-product based at least in part on the first vector and the second vector; and
a second multiplier configurable to determine a second partial inner-product based at least in part on the second vector and the third vector; and
adder circuitry configurable to determine a first inner-product of a dot-product based at least in part on the first partial inner-product; and
a second integrated circuit device communicatively coupled to the first integrated circuit device.
19. The system of claim 18 , wherein:
the first integrated circuit device comprises a programmable logic device; and
the second integrated circuit device comprises a processor.
20. The system of claim 18 , comprising a substrate, wherein the first integrated circuit device and the second integrated circuit device are mounted to the substrate.