Prefetching weights for use in a neural network processor

1. A circuit for performing neural network computations for a neural network comprising a plurality of layers, the circuit comprising:

a hardware matrix computation unit comprising circuitry for a two-dimensional array of cells, each cell configured to store weight inputs received from a source external to the cell;

first hardware circuitry configured to, for each of the plurality of neural network layers:

send, for the neural network layer, a plurality of weight inputs to the array of cells; and

second hardware circuitry configured to, for each of the plurality of neural network layers:

provide instructions along a first dimension of the two-dimensional array of cells to control operations performed by the cells along the first dimension, the operations being performed using the weight inputs and activation inputs shifted along a second dimension of the two-dimensional array of cells.

2. The circuit of claim 1 , wherein each cell of the two-dimensional array of cells includes a weight register configured to:

store one or more of the weight inputs that are shifted along the second dimension of the two-dimensional array of cells.

3. The circuit of claim 2 , wherein each cell of the two-dimensional array of cells is configured to:

compute, using multiplication circuitry in the cell, a product of an activation input and a respective weight input shifted along the second dimension of the two-dimensional array of cells.

4. The circuit of claim 3 , wherein each cell of the two-dimensional array of cells is configured to:

receive a control value based on the instructions provided along the first dimension of the two-dimensional array of cells; and

based on the control value, determine whether to store, in the weight register of the cell, the respective weight input shifted along the second dimension of the two-dimensional array of cells.

5. The circuit of claim 1 , wherein the circuit is configured to, for each of the plurality of neural network layers:

generate clock signals;

send control signals to each of the cells in the two-dimensional array of cells based on a timing of the clock signals; and

based on the control signals, execute, at different times and at different cells, a portion of the instructions provided along the first dimension of the two-dimensional array of cells.

6. The circuit of claim 5 , wherein the first hardware circuitry is configured to:

based on the control signals and the timing of the clock signals, shift the weight inputs and activation inputs to different cells along the second dimension of the two-dimensional array of cells.

7. The circuit of claim 1 , wherein the circuit comprises a processor configured to:

based on the instructions provided along the first dimension of the two-dimensional array of cells, cause a respective weight input to be loaded in a corresponding cell in the matrix computation unit, the respective weight input being among the weight inputs that are shifted along the second dimension of the two-dimensional array of cells.

8. The circuit of claim 7 , wherein the processor is further configured to:

based on the instructions, cause a respective weight input to be loaded at each cell of the two-dimensional array of cells over the plurality of clock cycles.

9. The circuit of claim 8 , wherein the processor is further configured to:

over the plurality of clock cycles, cause the respective weight input to be loaded at the cell and stored in a corresponding weight register at the cell.

10. The circuit of claim 1 , wherein the circuit is configured to:

prefetch the weight inputs; and

based on the prefetching of the weight inputs, preload the weight inputs at each of the different cells along the second dimension of the two-dimensional array of cells.

11. The circuit of claim 10 , wherein the weight inputs are preloaded before the activation inputs are shifted along the second dimension of the two-dimensional array of cells.

12. The circuit of claim 1 , wherein:

the first dimension of the two-dimensional array of cells corresponds to columns of the array; and

the second dimension of the two-dimensional array of cells corresponds to rows of the array.

13. A method for performing neural network computations for a neural network comprising a plurality of layers and using a circuit comprising a matrix computation unit, the method comprising:

for at least one layer of the plurality of neural network layers:

providing instructions along a first dimension of a two-dimensional array of cells in the matrix computation unit to control operations performed by the cells;

sending a plurality of weight inputs to cells along a second dimension of the two-dimensional array of cells, each of the cells configured to store weight inputs received from a source external to the cell;

sending activation inputs along the second dimension of the two-dimensional array of cells; and

based on the instructions, performing, by each of the cells in the two-dimensional array of cells, operations using the activation inputs and weight inputs stored at the cells.

14. The method of claim 13 , wherein each cell of the two-dimensional array of cells includes a weight register and the method further comprises:

storing, in a respective weight register of the cells, one or more of the weight inputs that are shifted along the second dimension of the two-dimensional array of cells.

15. The method of claim 14 , wherein each cell of the two-dimensional array of cells comprises respective multiplication circuitry and the method further comprises:

computing, using the respective multiplication circuitry in the cells, a product of an activation input and a respective weight input shifted along the second dimension of the two-dimensional array of cells.

16. The method of claim 15 , further comprising:

receiving, by a cell of the two-dimensional array of cells, a control value based on the instructions provided along the first dimension of the two-dimensional array of cells; and

based on the control value, determining whether to store, in the respective weight register of the cell, the respective weight input shifted along the second dimension of the two-dimensional array of cells.

17. The method of claim 13 , wherein for each of the plurality of neural network layers, the method further comprises:

generating clock signals;

sending control signals to each of the cells in the two-dimensional array of cells based on a timing of the clock signals; and

based on the control signals, executing, at different times and at different cells, a portion of the instructions provided along the first dimension of the two-dimensional array of cells.

18. The method of claim 17 , further comprising:

based on the control signals and the timing of the clock signals, shifting the weight inputs and activation inputs to different cells along the second dimension of the two-dimensional array of cells.

19. The method of claim 13 , wherein the circuit comprises a processor and the method further comprises:

based on the instructions provided along the first dimension of the two-dimensional array of cells, causing, by the processor, a respective weight input to be loaded in a corresponding cell in the matrix computation unit, the respective weight input being among the weight inputs that are shifted along the second dimension of the two-dimensional array of cells.

20. The method of claim 19 , further comprising:

based on the instructions, causing, by the processor, a respective weight input to be loaded at each cell of the two-dimensional array of cells over the plurality of clock cycles.

21. The method of claim 20 , further comprising:

over the plurality of clock cycles, causing, by the processor, the respective weight input to be loaded at the cell and stored in a corresponding weight register at the cell.

22. The method of claim 13 , further comprising:

prefetching the weight inputs; and

based on the prefetching of the weight inputs, preloading the weight inputs at each of the different cells along the second dimension of the two-dimensional array of cells;

wherein the weight inputs are preloaded before the activation inputs are shifted along the second dimension of the two-dimensional array of cells.

23. A non-transitory machine-readable storage device for storing instructions for performing neural network computations for a neural network comprising a plurality of neural network layers and using a circuit comprising a matrix computation unit, wherein the instructions are executable by a processing device to cause performance of operations comprising:

for at least one layer of the plurality of neural network layers:

providing instructions along a first dimension of a two-dimensional array of cells in the matrix computation unit to control operations performed by the cells;

sending a plurality of weight inputs to cells along a second dimension of the two-dimensional array of cells, each of the cells configured to store weight inputs received from a source external to the cell;

sending activation inputs along the second dimension of the two-dimensional array of cells; and

based on the instructions, performing, by each of the cells in the two-dimensional array of cells, operations using the activation inputs and weight inputs stored at the cells.