Computing circuit and data processing method based on convolutional neural network and computer readable storage medium
A computing circuit and a data processing method based on a convolutional neural network and a computer readable storage medium are provided. Input data is obtained from a memory. A first computation is performed on first part data of the input data to obtain first output data. The first output data is buffered in a first buffer. When the buffered first output data is greater than a first predetermined data amount, a second computation is performed on the first output data to obtain second output data. The second output data is buffered in a second buffer. Third output data obtained by performing a third computation on the second output data is output to the memory. When performing the second computation on the first output data, the first computation is continuously performed on the input data. Accordingly, the number of accesses of the memory can be reduced.
1. A data processing method based on a convolutional neural network (CNN), comprising:
reading, through a processing element, input data from a memory;
performing, through the processing element, a first computation on first part data of the input data to obtain first output data, wherein a CNN structure is configured with the first computation, a second computation, and a third computation, the first computation is configured with a first filter, and a size of the first output data is related to a size of the first filter of the first computation and a size of the first part data;
buffering, through the processing element, the first output data to a first buffer;
performing, through the processing element, the second computation on the first output data to obtain second output data when the first output data buffered in the first buffer is greater than a first predetermined data amount, wherein the second computation is configured with a second filter, and a size of the second output data is related to a size of the second filter of the second computation;
buffering, through the processing element, the second output data to a second buffer; and
outputting, through the processing element, third output data obtained by performing the third computation on the second output data to the memory, wherein
when performing the second computation on the first output data, the first computation is continuously performed on the input data, wherein
a height of the second filter is H kd , a width of the second filter is W kd , a height of the first output data is H f1o , a width of the first output data is W f1o , and H kd , W kd , H f1o , and W f1o are positive integers;
a height formed by a maximum data amount buffered in the first buffer is M H ×H f1o and a width formed is M W ×W f1o , and M H and M W are multiples and positive integers; and
when the first output data buffered in the first buffer is greater than the first predetermined data amount, M H ×H f1o is not less than H kd , and M W ×W f1o is not less than W kd .
2. The data processing method based on the convolutional neural network according to claim 1 , wherein the second computation is different from the third computation, and buffering the second output data to the second buffer comprises:
performing the third computation on the second output data when the second output data buffered in the second buffer is greater than a second predetermined data amount to obtain the third output data, wherein the third computation is configured with a third filter, and a size of the third output data is related to a size of the third filter.
3. The data processing method based on the convolutional neural network according to claim 2 , wherein outputting the third output data obtained by performing the third computation on the second output data to the memory comprises:
continuously performing the first computation and the second computation when performing the third computation on the second output data.
4. The data processing method based on the convolutional neural network according to claim 2 , wherein the third computation is a pointwise convolution computation, a height and a width of the third filter are both 1 , and performing the third computation on the third input data comprises:
performing the third computation on each of the buffered second output data.
5. The data processing method based on the convolutional neural network according to claim 2 , wherein performing the first computation on the first part data of the input data, performing the second computation on the first output data, or performing the third computation on the second output data comprises:
dividing the first part data into a plurality of first systolic array inputs; and
respectively performing the first computation on the first systolic array inputs to obtain a plurality of first systolic array outputs, wherein the first systolic array outputs form the first output data.
6. The data processing method based on the convolutional neural network according to claim 1 , wherein the second computation is a depthwise convolution computation, and performing the second computation on the first output data comprises:
performing the second computation when the first output data buffered in the first buffer is greater than W kd ×H kd .
7. The data processing method based on the convolutional neural network according to claim 6 , wherein the first computation is a convolution computation, and reading the input data from the memory comprises:
reading the first part data of the input data toward a first sliding direction, wherein the first sliding direction is parallel to a height of the input data.
8. The data processing method based on the convolutional neural network according to claim 6 , wherein the first computation is a convolution computation, and reading the input data from the memory comprises:
reading the first part data of the input data toward a second sliding direction, wherein the second sliding direction is parallel to a width of the input data.
9. A non-transitory computer readable storage medium, used to store a program code, wherein a processor loads the program code to execute the data processing method based on the convolutional neural network according to claim 1 .
10. A computing circuit based on a convolutional neural network, comprising:
a memory, used to store input data;
a processing element, coupled to the memory and comprising:
a first computing device, used to perform a first computation on first part data of the input data to obtain first output data, and buffer the first output data to a first buffer memory of the processing element, wherein a CNN structure is configured with the first computation, a second computation, and a third computation, the first computation is configured with a first filter, and a size of the first output data is related to a size of the first filter of the first computation and a size of the first part data;
a second computing device, used to perform the second computation on the first output data when the first output data buffered in the first buffer memory is greater than a first predetermined data amount to obtain second output data, and buffer the second output data to a second buffer memory, wherein the second computation is configured with a second filter, and a size of the second output data is related to a size of the second filter of the second computation;
the second buffer memory, used to store the second output data; and
a third computing device, used to output third output data obtained by performing the third computation on the second output data to the memory, wherein
when the second computing device performs the second computation, the first computing device continuously performs the first computation, wherein
a height of the second filter is H kd , a width of the second filter is W kd , a height of the first output data is H f1o , a width of the first output data is W f1o , and H kd , W kd , H f1o , and W f1o are positive integers;
a height formed by a maximum data amount buffered in the first buffer is M H ×H f1o and a width formed is M W ×W f1o , and M H and M W are multiples and positive integers; and
when the first output data buffered in the first buffer is greater than the first predetermined data amount, M H ×H f1o is not less than H kd , and M W ×W f1o is not less than W kd .
11. The computing circuit based on the convolutional neural network according to claim 10 , wherein the first computing device has a first maximum computation amount in a unit time, the second computing device has a second maximum computation amount in the unit time, the third computing device has a third maximum computation amount in the unit time, the first maximum computation amount is greater than the second maximum computation amount, and the first maximum computation amount is greater than the third maximum computation amount.
12. The computing circuit based on the convolutional neural network according to claim 10 , wherein when the third computing device runs the third computation, the first computing device continuously run the first computation, and the second computing device continuously run the second computation.
13. The computing circuit based on the convolutional neural network according to claim 10 , wherein the first buffer memory and the second buffer memory are static random access memories.