Granted Patent B2

US 12,455,737 · App. 18/505,743 · Granted Oct 28, 2025

Inventors: Olivier Temam (Antony, FR); Ravi Narayanaswami (San Jose, CA); Harshit Khaitan (San Jose, CA); Dong Hyuk Woo (San Jose, CA)

Assignee: Google LLC

G06F9/3001G06F9/30036G06F9/30065G06F9/3824G06F13/28G06N3/04G06N3/045G06N3/063

Abstract

Claims (37)

1. A hardware integrated circuit configured to implement a neural network, the integrated circuit comprising:

a first memory configured to store a first operand;

a second memory configured to store a second operand;

a linear computing unit comprising a plurality of multiplication cells having inputs that are coupled to the first memory and the second memory;

an output activation pipeline coupled to outputs of the plurality of multiplication cells, wherein the output activation pipeline includes at least one pipelined shift register; and

a non-linear unit coupled between the first memory and the at least one pipelined shift register, wherein the non-linear unit is configured to:

apply an activation function to the output of the linear computing unit that is shifted out of the at least one pipelined shift register, wherein the output comprises an accumulated value corresponding to a product of multiplying the first operand with the second operand using a multiplication cell of the linear computing unit; and

write, to the first memory, activations resulting from the applying of the activation function to the output of the linear computing unit.

2. The integrated circuit of claim 1 , wherein each of the plurality of multiplication cells of the linear computing unit is configured to:

process the first operand through a first layer of the neural network at least by multiplying the first operand with the second operand using the multiplication cell of the linear computing unit.

3. The integrated circuit of claim 1 , wherein the non-linear unit is configured to:

i) generate an output activation in response to applying the activation function to the accumulated value; and

ii) shift the output activation from the non-linear unit towards the first memory.

4. The integrated circuit of claim 1 , wherein the non-linear unit aggregates partial sums of the accumulated value into a final linear output based on a control signal provided to the non-linear unit.

5. The integrated circuit of claim 1 , wherein at least a portion of the plurality of multiplication cells are grouped to form a cell group of the linear computing unit that computes an output feature map.

6. The integrated circuit of claim 5 , wherein each cell of the cell group computes a subset of outputs in the output feature map.

7. The integrated circuit of claim 1 , wherein:

i) the first memory comprising a first memory bandwidth of less than 16-bits; and

ii) the second memory is a wide memory comprising a second memory bandwidth equal to or less than 32-bits.

8. The integrated circuit of claim 7 , wherein the first memory bandwidth is less than the second memory bandwidth.

9. A method for accelerating tensor computations using a hardware integrated circuit configured to implement a neural network, the method comprising:

receiving instructions from a source external to a non-linear unit of the integrated circuit, wherein the non-linear unit is coupled between a first memory and an output activation pipeline coupled to outputs of a linear computing unit comprising a plurality of multiplication cells, wherein the output activation pipeline includes at least one pipelined shift register;

multiplying, using a multiplication cell of the linear computing unit, a first operand obtained from the first memory with a second operand obtained from a second memory of the integrated circuit; and

applying, by the non-linear unit, the output of the linear computing unit that is shifted out of the at least one pipelined shift register, wherein the output comprises an accumulated value corresponding to a product of multiplying the first operand with the second operand using a multiplication cell of the linear computing unit; and

writing, to the first memory, activations resulting from the applying of the activation function to the output of the linear computing unit.

10. The method of claim 9 , further comprising:

processing the first operand through a first layer of the neural network at least by multiplying the first operand with the second operand using the multiplication cell of the linear computing unit.

11. The method of claim 9 , further comprising:

generating, by the non-linear unit, an output activation in response to applying the activation function to the accumulated value; and

shifting the output activation from the non-linear unit towards the first memory.

12. The method of claim 9 , wherein the non-linear unit aggregates partial sums of the accumulated value into a final linear output based on a control signal provided to the non-linear unit.

13. The method of claim 9 , wherein at least a portion of the plurality of multiplication cells are grouped to form a cell group of the linear computing unit that computes an output feature map.

14. The method of claim 13 , wherein each cell of the cell group computes a subset of outputs in the output feature map.

15. The method of claim 9 , wherein:

i) the first memory comprising a first memory bandwidth of less than 16-bits; and

ii) the second memory is a wide memory comprising a second memory bandwidth equal to or less than 32-bits.

16. The method of claim 15 , wherein the first memory bandwidth is less than the second memory bandwidth.

Assignments (2)

Continuity (4)

References Cited (192)