Neural network operation method and apparatus with stacking-based channel alignment
A neural network operation method and apparatus are disclosed, where the network operation method including receiving data for a neural network operation, determining whether a size of the data is less than or equal to a threshold, generating stacked data by stacking a portion of the data based on the determining, and performing the neural network operation in parallel based on the stacked data.
1 . A neural network operation method comprising:
receiving data for a neural network operation, the data comprising a first map that is a feature map comprising first valid channels, and comprising a second map that is a feature map comprising second valid channels;
determining whether a size of the data is less than or equal to a threshold;
in response to a determination that the size of the data is less than or equal to the threshold, generating stacked data based on combining at least a portion of the first map with at least a portion of the second map such that a last valid channel of the first map is adjacent to a first valid channel of the second map in the stacked data; and
performing a first convolution operation for the first map and a second convolution operation for the second map in parallel using the stacked data; and
generating, in parallel, a first output of a result of the first convolution operation and a second output of a result of the second convolution operation, wherein a number of outputs generated based on a convolution operation on the stacked data is equal to a number of feature maps used to generate the stacked data.
2 . The neural network operation method of claim 1 , wherein the performing comprises performing a convolution operation between the stacked data and a stacked kernel corresponding to the stacked data.
3 . The neural network operation method of claim 1 , wherein the generating of the stacked data comprises:
generating tiles by segmenting the data to have a predetermined width or a predetermined height; and
generating the stacked data by stacking the tiles.
4 . The neural network operation method of claim 1 , wherein the generating of the stacked data comprises generating the stacked data by inputting the data to a direct memory access (DMA) engine.
5 . The neural network operation method of claim 1 , wherein the generating of the stacked data further comprises:
searching for additional data to perform a second neural network operation that is different from the neural network operation performed based on the data;
determining whether the additional data and the data are stackable; and
performing the neural network operation and the second neural network operation in parallel by stacking the additional data and the data based on a result of the determining whether the additional data and the data are stackable.
6 . A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .
7 . A neural network operation apparatus comprising:
one or more processors; and
a memory storing instructions that when executed by the one or more processors cause the neural network operation apparatus to:
determine whether a size of the data for a neural network operation is less than or equal to a threshold, the data comprising a first map that is a feature map comprising first valid channels, and comprising a second map that is a feature map comprising second valid channels,
in response to a determination that the size of the data is less than or equal to the threshold, generate stacked data based on combining at least a portion of the first map with at least a portion of the second map such that a last valid channel of the first map is adjacent to a first valid channel of the second map,
perform a first convolution operation for the first map and a second convolution operation for the second map in parallel using the stacked data, and
generate, in parallel, a first output of a result of the first convolution operation and a second output of a result of the second convolution operation, wherein a number of outputs generated based on a convolution operation on the stacked data is equal to a number of feature maps used to generate the stacked data.
8 . The neural network operation apparatus of claim 7 , wherein the instructions are further configured to cause the neural network operation apparatus to perform a convolution operation between the stacked data and a stacked kernel corresponding to the stacked data.
9 . The neural network operation apparatus of claim 7 , wherein the instructions are further configured to cause the neural network operation apparatus to:
generate tiles by segmenting the data to have a predetermined width or a predetermined height, and
generate the stacked data by stacking the tiles.
10 . The neural network operation apparatus of claim 7 , wherein the instructions are further configured to cause the neural network operation apparatus to generate the stacked data by inputting the data to a direct memory access (DMA) engine.
11 . The neural network operation method of claim 1 , wherein, in the stacked data, the first valid channel is self-contiguous and the second valid channel is self-contiguous.
12 . The neural network operation apparatus of claim 7 , wherein, in the stacked data, the first valid channel is self-contiguous and the second valid channel is self-contiguous.