Inner product convolutional neural network accelerator
A convolutional neural network (CNN) accelerator, including: a CNN circuit for performing a multiple-layer CNN computation, wherein the multiple layers are to receive an input feature according to an input feature map (IFM) and a weight matrix per output feature, wherein an output of a first layer provides an input for a next layer; and a mapping circuit to access a three-dimensional input matrix stored as a Z-major matrix; wherein the CNN circuit is to perform an inner-product direct convolution on the Z-major matrix, wherein the direct convolution lacks a lowering operation.
1. A convolutional neural network (CNN) accelerator, comprising:
a CNN circuit for performing computation in a CNN layer of a CNN, wherein the CNN layer is to receive a three-dimensional input matrix and to generate an output; and
a mapping circuit configured to:
access the three-dimensional input matrix from a first memory, wherein the three-dimensional input matrix comprises data elements in a X-Y-Z data structure, the data elements of the three-dimensional input matrix are stored in a first storage order, and a layout of the first memory comprises a first memory location, in which data elements having different X coordinates and a same Y or Z coordinate are stored continuously, and a second memory location, in which data elements having different X coordinates and another same Y or Z coordinate are stored continuously,
convert the first storage order of the data elements to a second storage order that is different from the first storage order, and
write the three-dimensional input matrix into a second memory in accordance with the second storage order, wherein a layout of the second memory comprises a first memory location, in which data elements having different Z coordinates and a same X or Y coordinate are stored continuously, and a second memory location, in which data elements having different Z coordinates but another same X or Y coordinate are stored continuously,
wherein the CNN circuit is configured to perform the computation in the CNN layer on the three-dimensional input matrix retrieved from the second memory.
2. The CNN accelerator of claim 1 , wherein the computation in the CNN layer comprises an inner-product convolution that is a vector-by-vector operation.
3. The CNN accelerator of claim 1 , wherein converting the first storage order of the data elements of the three-dimensional input matrix to the second storage order comprises rotating two-dimensional matrices in the three-dimensional input matrix on X, Y, or Z axis.
4. The CNN accelerator of claim 1 , wherein the CNN is a low-precision CNN.
5. The CNN accelerator of claim 4 , wherein the low-precision CNN is a 4-bit CNN.
6. The CNN accelerator of claim 4 , wherein the low-precision CNN is a ternary CNN.
7. The CNN accelerator of claim 4 , wherein the low-precision CNN is a binary CNN.
8. The CNN accelerator of claim 4 , wherein the low-precision CNN comprises a high-precision input feature map with a low-precision weight, and the weight precision is 1-bit, 2-bit, or 4-bit.
9. The CNN accelerator of claim 1 , wherein the CNN is a high-precision CNN.
10. The CNN accelerator of claim 9 , wherein the high-precision CNN is selected from 8-bit integer and 16-bit floating point.
11. The CNN accelerator of claim 1 , further comprising at least one accumulator, wherein the results of multiple inner-product convolutions are accumulated in a single accumulator.
12. The CNN accelerator of claim 1 , further comprising a reduction module, the reduction module to:
receive a plurality of high-precision input operators; and
change the plurality of high-precision input operators to a low-precision output operator.
13. The CNN accelerator of claim 1 , further comprising a reduction module, the reduction module comprising:
a subtract circuit to compute the difference between a value and a first input operand to yield a first intermediate operand;
a pass-through of a second input operand and a third input operand to yield a second intermediate operand and a third intermediate operand, respectively;
a multiply circuit to compute a product of a fourth input operand and a fifth input operand to yield a fourth intermediate input operand; and
a reduction circuit to compute an output operator by computing a reduction of the first intermediate operand, the second intermediate operand, the third intermediate operand, and the fourth intermediate operand.
14. The CNN accelerator of claim 13 , wherein the first input operand, the second input operand, the third input operand, the fourth input operand, and the fifth input operand are half-precision floating point operands, and the output operator is a 1-bit or 2-bit integer operand.
15. The CNN accelerator of claim 13 , wherein the first input operation, the fourth input operand, and the fifth input operation are configurable operands.
16. The CNN accelerator of claim 13 , wherein the reduction circuit comprises a bit-select operator, and the output operator is selected from one or two digits of an accumulator.
17. The CNN accelerator of claim 16 , wherein the output operator is selected from a sign digit of the accumulator.
18. The CNN accelerator of claim 13 , wherein the reduction circuit comprises a quantization operator, and the output operator is selected based on a comparison of two or more of the intermediate operators.
19. One or more tangible, non-transitory computer-readable storage mediums storing instructions executable to:
receive a three-dimensional input matrix of a CNN layer of a CNN;
access the three-dimensional input matrix from a first memory, wherein the three-dimensional input matrix comprises data elements in a X-Y-Z data structure, the data elements of the three-dimensional input matrix are stored in a first storage order, and a layout of the first memory comprises a first memory location, in which data elements having different X coordinates and a same Y or Z coordinate are stored continuously, and a second memory location, in which data elements having different X coordinates and another same Y or Z coordinate are stored continuously
convert the first storage order of the data elements to a second storage order that is different from the first storage order;
write the three-dimensional input matrix into a second memory in accordance with the second storage order, wherein a layout of comprises a first memory location, in which data elements having different Z coordinates and a same X or Y coordinate are stored continuously, and a second memory location, in which data elements having different Z coordinates but another same X or Y coordinate are stored continuously; and
perform an inner-product direct convolution in the CNN layer on the three-dimensional input matrix retrieved from the second memory.
20. The one or more tangible, non-transitory computer-readable mediums of claim 19 , wherein the inner-product convolution is a vector-by-vector operation.
21. The one or more tangible, non-transitory computer-readable mediums of claim 19 , wherein converting the first storage order of the data elements of the three-dimensional input matrix to the second storage order comprises rotating two-dimensional matrices in the three-dimensional input matrix on X, Y, or Z axis.
22. The one or more tangible, non-transitory computer-readable mediums of claim 19 , wherein the CNN is a low-precision CNN.
23. The one or more tangible, non-transitory computer-readable mediums of claim 22 , wherein the low-precision CNN is a ternary CNN or binary CNN.
24. A computer-implemented method comprising:
receiving a three-dimensional input matrix of a CNN layer of a CNN
accessing the three-dimensional input matrix from a first memory, wherein the three-dimensional input matrix comprises data elements in a X-Y-Z data structure, the data elements of the three-dimensional input matrix are stored in a first storage order, and a layout of the first memory comprises a first memory location, in which data elements having different X coordinates and a same Y or Z coordinate are stored continuously, and a second memory location, in which data elements having different X coordinates but another same Y or Z coordinate are stored continuously;
converting the first storage order of the data elements to a second storage order that is different from the first storage order;
writing the three-dimensional input matrix into a second memory in accordance with the second storage order, wherein a layout of the second memory comprises a first memory location, in which data elements having different Z coordinates and a same X or Y coordinate are stored continuously, and a second memory location, in which data elements having different Z coordinates but another same X or Y coordinate are stored continuously; and
performing an inner-product direct convolution in the CNN layer on the three-dimensional input matrix retrieved from the second memory.
25. The computer-implemented method of claim 24 , wherein converting the first storage order of the data elements of the three-dimensional input matrix to the second storage order comprises rotating first two-dimensional matrices in the three-dimensional input matrix on X, Y, or Z axis.