Granted Patent B2

US 12,730,634 · App. 19/024,955 · Granted Sep 8, 2026

Inventors: Olivier Temam (Antony, FR); Ravi Narayanaswami (San Jose, CA); Harshit Khaitan (San Jose, CA); Dong Hyuk Woo (San Jose, CA)

Assignee: Google LLC

G06F9/3001G06F9/30036G06F9/30065G06F9/3824G06F13/28G06N3/04G06N3/063

Abstract

Claims (30)

1 . A machine learning accelerator comprising a tensor traversal unit (TTU) configured to track a respective state of a plurality tensors being processed,

wherein the TTU comprises, for each tensor of the plurality of tensors being processed:

a plurality of counter registers,

a plurality of stride registers, and

a plurality of limit registers,

wherein the TTU is configured to perform operations comprising:

generating an address for a particular tensor element of a tensor based on values stored in respective counter registers, stride registers, and limit registers for the tensor, wherein the machine learning accelerator is configured to obtain one or more tensor values from memory based on the generated address;

receiving an indication of an element of a tensor being committed to the memory; and

in response, incrementing one or more counter registers for the tensor.

2 . The machine learning accelerator of claim 1 , wherein the machine learning accelerator is configured to execute a plurality of different instruction types, and wherein the machine learning accelerator has a dedicated TTU for each instruction type of the plurality of instruction types.

3 . The machine learning accelerator of claim 2 , wherein the machine learning accelerator has a direct-memory-access TTU for a first type of instruction type and a control TTU for a second type of instruction.

4 . The machine learning accelerator of claim 1 , wherein a number of counter registers for each tensor corresponds to a depth of the tensor.

5 . The machine learning accelerator of claim 1 , wherein the machine learning accelerator comprises a narrow memory bank and a wide memory bank, and wherein the TTU is configured to generate an address into the narrow memory bank, the wide memory bank, or both.

6 . The machine learning accelerator of claim 5 , wherein the narrow memory bank is configured to store input activations, and wherein the wide memory bank is configured to store weights of a particular layer of a neural network.

7 . The machine learning accelerator of claim 1 , wherein the TTU is configured to generate multiple addresses on subsequent iterations while the machine learning accelerator executes a single instruction.

8 . A method performed by a machine learning accelerator comprising a tensor traversal unit (TTU) configured to track a respective state of a plurality tensors being processed,

wherein the TTU comprises, for each tensor of the plurality of tensors being processed:

a plurality of counter registers,

a plurality of stride registers, and

a plurality of limit registers,

wherein the method comprises:

generating an address for a particular tensor element of a tensor based on values stored in respective counter registers, stride registers, and limit registers for the tensor, wherein the machine learning accelerator is configured to obtain one or more tensor values from memory based on the generated address;

receiving an indication of an element of a tensor being committed to the memory; and

in response, incrementing one or more counter registers for the tensor.

9 . The method of claim 8 , further comprising executing a plurality of different instruction types, and wherein the machine learning accelerator has a dedicated TTU for each instruction type of the plurality of instruction types.

10 . The method of claim 9 , wherein the machine learning accelerator has a direct-memory-access TTU for a first type of instruction type and a control TTU for a second type of instruction.

11 . The method of claim 8 , wherein a number of counter registers for each tensor corresponds to a depth of the tensor.

12 . The method of claim 8 , wherein the machine learning accelerator comprises a narrow memory bank and a wide memory bank, and wherein the method further comprises generating an address into the narrow memory bank, the wide memory bank, or both.

13 . The method of claim 12 , wherein the narrow memory bank is configured to store input activations, and wherein the wide memory bank is configured to store weights of a particular layer of a neural network.

14 . The method of claim 8 , further comprising generating multiple addresses on subsequent iterations while the machine learning accelerator executes a single instruction.

Assignments (3)

Continuity (5)

References Cited (194)