Rotating data for neural network computations
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computing a layer output for a convolutional neural network layer, the method comprising: receiving a plurality of activation inputs; forming a plurality of vector inputs from the plurality of activation inputs, each vector input comprising values from a distinct region within the multi-dimensional matrix; sending the plurality of vector inputs to one or more cells along a first dimension of the systolic array; generating a plurality of rotated kernel structures from each of the plurality of kernel; sending each kernel structure and each rotated kernel structure to one or more cells along a second dimension of the systolic array; causing the systolic array to generate an accumulated output based on the plurality of value inputs and the plurality of kernels; and generating the layer output from the accumulated output.
1. A method for performing computations for a layer of a neural network using a hardware array of compute cells in a hardware integrated circuit configured to implement the neural network, the method comprising:
receiving an input data that flows through the hardware integrated circuit along a particular dimension of the hardware array of compute cells;
obtaining a multi-dimensional matrix structure comprising operands of the input data by reading the operands from a memory of the integrated circuit; and
performing a convolution between operands of the multi-dimensional matrix structure and a kernel for the neural network layer, comprising:
rotating the multi-dimensional matrix structure across a first dimension of the hardware array; and
performing, by the hardware array, the convolution using operands from the rotated multi-dimensional matrix structure and the kernel for the neural network layer.
2. The method of claim 1 , wherein the hardware array comprises an array of multiplication circuits and performing the convolution comprises:
performing, by the array of multiplication circuits, multiplication operations between operands of the multi-dimensional matrix structure and a weight input of the kernel.
3. The method of claim 1 , wherein:
the multi-dimensional matrix structure is a matrix structure of activation inputs; and
the kernel is one of a plurality of kernels, each of the plurality of kernels being a matrix structure of weight inputs for one or more layers of the neural network.
4. The method of claim 3 , further comprising:
converting the matrix structure of activation inputs to a first permuted matrix structure; and
converting at least one matrix structure of weight inputs to a second permuted matrix structure.
5. The method of claim 1 , wherein performing, by the hardware array, the convolution using operands from the rotated multi-dimensional matrix structure and the kernel for the neural network layer produces a plurality of accumulation values, wherein the method comprises applying an activation function to the plurality of accumulated values to produce a layer output.
6. The method of claim 5 , comprising storing the layer output in a buffer as an input to a subsequent layer in the neural network.
7. The method of claim 1 , comprising receiving instructions at a host interface, wherein the instructions comprise one or more of a number of layers of the neural network to process, corresponding input and output sizes for each layer of the neural network, a stride value, and a type of layer to be processed.
8. The method of claim 7 , comprising sending the instructions to a sequencer, wherein the sequencer converts the instructions into control signals.
9. The method of claim 8 , comprising sending at least one control signal from the sequencer to a direct memory access engine.
10. The method of claim 8 , comprising generating, by the sequencer, clocked signals.
11. The method of claim 1 , wherein rotating the multi-dimensional matrix structure comprises shifting a first set of operands in a first value loader to a second value loader.
12. The method of claim 11 , wherein shifting occurs after a single clock cycle.
13. The method of claim 1 , comprising rotating the kernel.
14. The method of claim 13 , wherein performing the convolution comprises performing the convolution using the operands from the rotated multi-dimensional matrix structure and weights from the rotated kernel.
15. The method of claim 13 , wherein performing the convolution comprises performing the convolution using the operands from the rotated multi-dimensional matrix structure and a subset of the rotated kernel.