Neural network instruction set architecture

1. A method for accelerating tensor computations for a neural network implemented on a hardware integrated circuit and having a plurality of neural network layers, the method comprising:

obtaining, using a controller of the hardware integrated circuit, a single instruction encoding at least one opcode that specifies a layer type of a neural network layer;

providing, by the controller, the single instruction to each of a plurality of compute units that are individual compute tiles of the hardware integrated circuit;

for each of the plurality of compute units: the single instruction, when executed by a respective processor of the compute unit, causes the compute unit to perform operations comprising:

determining that the tensor computations are for a particular type of neural network layer based on an opcode in the single instruction that specifies the layer type of the neural network layer from a plurality of neural network layer types that includes: a convolutional layer type and a fully connected layer type;

determining, based on the single instruction and the layer type of the neural network layer, a compute sequence for executing a respective set of operations to process data values at a portion of an input tensor through the neural network layer; and

performing, based on the compute sequence and the single instruction, the tensor computations for the particular type of neural network layer using inputs and weights obtained from memory address locations accessed at the compute unit, comprising:

traversing, based on the single instruction, elements along one or more dimensions of the input tensor using a respective hardware processing unit of the compute unit; and

generating, based on the single instruction, an output for the particular type of neural network layer in response to performing the tensor computations across the plurality of compute units.

2. The method of claim 1 , wherein the tensor computations performed at the compute unit are executed based on a loop nest comprising a plurality of processing loops, where a structure of the loop nest is defined based on opcodes in the single instruction and is specific to the particular type of neural network layer.

3. The method of claim 1 , wherein the operations further comprise:

receiving the inputs to a first neural network layer based on the single instruction being executed at the compute unit; and

storing the inputs at a first memory of the compute unit based on a first memory access operation that uses a first set of memory address locations.

4. The method of claim 3 , wherein the operations further comprise:

receiving the weights for the first neural network layer based on the single instruction being executed at the compute unit; and

storing the weights at a second memory of the compute unit based on a second memory access operation that uses a second set of memory address locations.

5. The method of claim 4 , wherein performing the tensor computations comprises:

obtaining an input from a memory location of the first memory;

obtaining a weight from a memory location of the second memory; and

performing the tensor computations using the input of the first memory and the weight of the second memory.

6. The method of claim 5 , wherein performing the tensor computations comprises:

generating an activation value using the input obtained from the first memory and the weight obtained from the second memory;

providing the activation value to a data bus configured to exchange tensor data between respective compute units; and

providing the activation value to a second compute unit using the data bus.

7. The method of claim 4 , wherein:

the first set of memory address locations represent elements of the input tensor; and

the second set of memory address locations represent elements of a weight tensor.

8. The method of claim 7 , wherein the operations further comprise:

determining a loop nest structure for generating a nested loop;

generating a plurality of nested loops using the loop nest structure; and

performing the tensor computations for the first neural network layer using the plurality of nested loops.

9. The method of claim 7 , wherein performing the tensor computations comprises:

using a tensor traversal unit to access memory address locations of the input tensor to obtain multiple inputs to the first neural network layer;

using the tensor traversal unit to access memory address locations of the weight tensor to obtain multiple weights for the first neural network layer; and

generating a dot product using the multiple inputs of the input tensor and the multiple weights of the weight tensor.

10. A system for accelerating tensor computations for a neural network implemented on a hardware integrated circuit and having a plurality of neural network layers, the system comprising:

a plurality of compute units that are individual compute tiles of the hardware integrated circuit;

a controller configured to: i) obtain a single instruction encoding at least one opcode that specifies a layer type of a neural network layer and ii) provide the single instruction to each of the plurality of compute units,

wherein, for each of the plurality of compute units: the single instruction, when executed by a respective processor of the compute unit, causes the compute unit to perform operations comprising:

determining that the tensor computations are for a particular type of neural network layer based on an opcode in the single instruction that specifies the layer type of the neural network layer from a plurality of neural network layer types that includes: a convolutional layer type and a fully connected layer type;

determining, based on the single instruction and the layer type of the neural network layer, a compute sequence for executing a respective set of operations to process data values at a portion of an input tensor through the neural network layer; and

performing, based on the compute sequence and the single instruction, the tensor computations for the particular type of neural network layer using inputs and weights obtained from memory address locations accessed at the compute unit, comprising:

traversing, based on the single instruction, elements along one or more dimensions of the input tensor using a respective hardware processing unit of the compute unit; and

generating, based on the single instruction, an output value for the particular type of neural network layer in response to performing the tensor computations at the compute unit.

11. The system of claim 10 , wherein the tensor computations performed at the compute unit are executed based on a loop nest comprising a plurality of processing loops, where a structure of the loop is defined based on opcodes in the single instruction and is specific to the particular type of neural network layer.

12. The system of claim 10 , wherein the operations further comprise:

receiving the inputs to a first neural network layer based on the single instruction being executed at the compute unit; and

storing the inputs at a first memory of the compute unit based on a first memory access operation that uses a first set of memory address locations.

13. The system of claim 12 , wherein the operations further comprise:

receiving the weights for the first neural network layer based on the single instruction being executed at the compute unit; and

storing the weights at a second memory of the compute unit based on a second memory access operation that uses a second set of memory address locations.

14. The system of claim 13 , wherein performing the tensor computations comprises:

obtaining an input from a memory location of the first memory;

obtaining a weight from a memory location of the second memory; and

performing the tensor computations using the input of the first memory and the weight of the second memory.

15. The system of claim 14 , wherein performing the tensor computations comprises:

generating an activation value using the input obtained from the first memory and the weight obtained from the second memory;

providing the activation value to a data bus configured to exchange tensor data between respective compute units; and

providing the activation value to a second compute unit using the data bus.

16. The system of claim 13 , wherein:

the first set of memory address locations represent elements of the input tensor; and

the second set of memory address locations represent elements of a weight tensor.

17. The system of claim 16 , wherein the operations further comprise:

determining a loop nest structure for generating a nested loop;

generating a plurality of nested loops using the loop nest structure; and

performing the tensor computations for the first neural network layer using the plurality of nested loops.

18. The system of claim 16 , wherein performing the tensor computations comprises:

using a tensor traversal unit to access memory address locations of the input tensor to obtain multiple inputs to the first neural network layer;

using the tensor traversal unit to access memory address locations of the weight tensor to obtain multiple weights for the first neural network layer; and

generating a dot product using the multiple inputs of the input tensor and the multiple weights of the weight tensor.

19. One or more non-transitory machine-readable storage devices for storing instructions used to accelerate tensor computations for a neural network implemented on a hardware integrated circuit and having a plurality of neural network layers, the instructions being executable by one or more processors to cause performance of operations comprising:

obtaining, using a controller of the hardware integrated circuit, a single instruction encoding at least one opcode that specifies a layer type of a neural network layer;

providing, by the controller, the single instruction to each of a plurality of compute units that are individual compute tiles of the hardware integrated circuit;

for each of the plurality of compute units: the single instruction, when executed by a respective processor of the compute unit, causes the compute unit to perform operations comprising:

determining that the tensor computations are for a particular type of neural network layer based on an opcode in the single instruction that specifies the layer type of the neural network layer from a plurality of neural network layer types that includes: a convolutional layer type and a fully connected layer type;

determining, based on the single instruction and the layer type of the neural network layer, a compute sequence for executing a respective set of operations to process data values at a portion of an input tensor through the neural network layer; and

performing, based on the compute sequence and the single instruction, the tensor computations for the particular type of neural network layer using inputs and weights obtained from memory address locations accessed at the compute unit, comprising:

traversing, based on the single instruction, elements along one or more dimensions of the input tensor using a responsive hardware processing unit of the compute unit; and

generating, based on the single instruction, an output for the particular type of neural network layer in response to performing the tensor computations across the plurality of compute units.

20. The machine-readable storage devices of claim 19 , wherein the operations further comprise:

receiving the inputs to the particular type of neural network layer based on the single instruction being executed at the compute unit, and storing the inputs at a first memory of the compute unit based on a first memory access operation that uses a first set of memory address locations; and

receiving the weights for the particular type of neural network layer based on the single instruction being executed at the compute unit, and storing the weights at a second memory of the compute unit based on a second memory access operation that uses a second set of memory address locations.