IP Library › Granted Patent US 12710926
Granted Patent B1
US 12710926 · App. 17/937,329 · Granted Aug 18, 2026

Integrated circuit for vector compute engine

Inventors: Paul Gilbert Meyer (Jericho, VT); Sundeep Amirineni (Cedar Park, TX); Ron Diamant (San Jose, CA)
Assignee: Amazon Technologies, Inc.
G06F7/57G06F2207/3884
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12710926
App. No.
17/937,329
Granted
Aug 18, 2026
Kind
B1
Abstract

An integrated circuit device for vector computations can include multiple computational circuit blocks coupled in series in a pipeline. Each computational circuit block can include an arithmetic logic unit (ALU) circuit having a first numeric input, a second numeric input, and a primary result output. The ALU circuit is programmable to perform a computational operation on the first numeric input and the second numeric input to generate the primary result output. Each computational circuit block can also include an output register to receive the primary result output of the ALU and provide the primary result output of the ALU circuit as a primary output of the computational circuit block, and one or more feedback registers to receive the primary result output of the ALU and provide the primary result output of the ALU circuit to a previous computational circuit block of the pipeline.

Claims (58)

1 . An integrated circuit device comprising:

a plurality of computational circuit blocks coupled in series in a pipeline, wherein each computational circuit block includes:

an arithmetic logic unit (ALU) circuit having a first numeric input, a second numeric input, and a primary result output, wherein the ALU circuit is programmable to perform a computational operation on the first numeric input and the second numeric input to generate the primary result output;

a first input multiplexor operable to select one of a plurality of inputs of the computational circuit block as the first numeric input to the ALU circuit;

a second input multiplexor operable to select one of the plurality of inputs of the computational circuit block as the second numeric input to the ALU circuit;

an output register configured to receive the primary result output of the ALU circuit and provide the primary result output of the ALU circuit as a primary output of the computational circuit block;

feedback circuitry including one or more feedback registers each configured to receive the primary result output of the ALU circuit and provide the primary result output of the ALU circuit to a previous computational circuit block of the pipeline; and

bypass circuitry including one or more bypass circuits each including a bypass multiplexor coupled to a bypass delay register, and operable to independently select one of the plurality of inputs of the computational circuit block to output to a next computational circuit block of the pipeline.

2 . The integrated circuit device of claim 1 , wherein the feedback circuitry of a computational circuit block includes multiple feedback registers to enable the computational circuit block to store multiple running computational results on different sets of data elements being streamed into the pipeline.

3 . The integrated circuit device of claim 1 , wherein the plurality of inputs of the computational circuit block includes a primary output of the previous computational circuit block of the pipeline, the primary output of the computational circuit block, an output of each feedback register of the next computational circuit block, and an output of each bypass circuit of the previous computational circuit block.

4 . The integrated circuit device of claim 3 ,

wherein each computational circuit block further includes a swap register configured to receive a secondary output of the ALU circuit, and feedback the secondary output of the ALU circuit to the computational circuit block, the secondary output being one of the first numeric input or the second numeric input, and

wherein the plurality of inputs of the computational circuit block further includes an output of the swap register.

5 . The integrated circuit device of claim 1 , wherein the pipeline includes at least eight ALU circuits coupled in series.

6 . An integrated circuit device comprising:

a plurality of computational circuit blocks coupled in series in a pipeline, wherein each computational circuit block includes:

an arithmetic logic unit (ALU) circuit having a first numeric input, a second numeric input, and a primary result output, wherein the ALU circuit is programmable to perform a computational operation on the first numeric input and the second numeric input to generate the primary result output;

an output register configured to receive the primary result output of the ALU circuit and provide the primary result output of the ALU circuit as a primary output of the computational circuit block;

a first feedback register configured to receive the primary result output of the ALU circuit and provide the primary result output of the ALU circuit to a previous computational circuit block of the pipeline; and

a second feedback register configured to receive the primary result output of the ALU circuit and provide the primary result output of the ALU circuit to the previous computational circuit block of the pipeline.

7 . The integrated circuit device of claim 6 , wherein each computational circuit block further includes:

a plurality of bypass circuits arranged in parallel, wherein each bypass circuit includes a bypass multiplexor coupled to a bypass delay register, and each bypass circuit is operable to independently select one of a plurality of inputs of the computational circuit block to output to a next computational circuit block of the pipeline.

8 . The integrated circuit device of claim 7 , wherein the plurality of inputs of the computational circuit block includes a primary output of the previous computational circuit block of the pipeline, the primary output of the computational circuit block, an output of a first feedback register of the next computational circuit block, an output of a second feedback register of the next computational circuit block, and an output of each bypass circuit of the previous computational circuit block.

9 . The integrated circuit device of claim 8 , wherein each computational circuit block further includes:

a first input multiplexor operable to select one of the plurality of inputs as the first numeric input to the ALU circuit; and

a second input multiplexor operable to select one of the plurality of inputs as the second numeric input to the ALU circuit.

10 . The integrated circuit device of claim 9 , wherein each computational circuit block further includes:

a swap register configured to receive a secondary output of the ALU circuit, and feedback the secondary output of the ALU circuit to the computational circuit block, the secondary output being one of the first numeric input or the second numeric input.

11 . The integrated circuit device of claim 10 , wherein the plurality of inputs selectable by at least one of the first and second input multiplexors includes an output of the swap register.

12 . The integrated circuit device of claim 6 , further comprising an input circuit operable to provide input data selectable from a plurality of data sources to the pipeline of computational circuit blocks, wherein the plurality of data sources includes:

a first tensor stored in a memory subsystem;

a second tensor stored in the memory subsystem; and

at least one of:

a parameter table;

a pseudo-random number generator; or

a counter.

13 . The integrated circuit device of claim 12 , wherein the plurality of data sources further includes:

a programmable constant value;

a zero value;

a floating-point one value;

an integer one value;

a positive infinity value; and

a negative infinity value.

14 . The integrated circuit device of claim 12 , wherein the plurality of data sources further includes a power-of-two value or a bit mask value.

15 . The integrated circuit device of claim 6 , wherein the first feedback register and the second feedback register have independent enable signals.

16 . The integrated circuit device of claim 15 , wherein the computational circuit block is programmable to sequentially enable the first feedback register alternately with the second feedback register.

17 . An integrated circuit device comprising:

a plurality of computational circuit blocks coupled in series in a pipeline, wherein each computational circuit block includes:

an arithmetic logic unit (ALU) circuit having a first numeric input, a second numeric input, and a primary result output, wherein the ALU circuit is programmable to perform a computational operation on the first numeric input and the second numeric input to generate the primary result output;

an output register configured to receive the primary result output of the ALU circuit and provide the primary result output of the ALU circuit as a primary output of the computational circuit block;

feedback circuitry including one or more feedback registers each configured to receive the primary result output of the ALU circuit and provide the primary result output of the ALU circuit to a previous computational circuit block of the pipeline; and

a plurality of bypass circuits arranged in parallel, wherein each bypass circuit includes a bypass multiplexor coupled to a bypass delay register, and each bypass circuit is operable to independently select one of a plurality of inputs of the computational circuit block to output to a next computational circuit block of the pipeline.

18 . The integrated circuit device of claim 17 , wherein each computational circuit block further includes:

a first input multiplexor operable to select one of the plurality of inputs as the first numeric input to the ALU circuit; and

a second input multiplexor operable to select one of the plurality of inputs as the second numeric input to the ALU circuit.

19 . The integrated circuit device of claim 18 , wherein the first input multiplexor or the second input multiplexor is operable to select an output of a bypass circuit of a previous computational circuit block as a numeric input to the ALU circuit.

20 . The integrated circuit device of claim 17 , wherein the bypass multiplexor is operable to select a primary output of a previous computational circuit block to provide to the next computational circuit block.

21 . The integrated circuit device of claim 17 , further comprising an output circuit coupled to a last computational circuit block of the pipeline, and operable to select between a primary output of the last computational circuit block and outputs of the bypass circuits of the last computation circuit block to write to memory.