Method and apparatus with convolution neural network processing using shared operand
A neural network apparatus including one or more processors including a controller configured to determine a shared operand to be shared in parallelized operations as being either one of a pixel value among pixel values of an input feature map and a weight value among weight values of a kernel, based on either one or both of a feature of the input feature map and a feature of the kernel, and one or more processing units configured to perform the parallelized operations based on the determined shared operand.
1. A neural network apparatus, comprising:
one or more processors configured to execute instructions; and
a memory storing the instructions, wherein execution of the instructions configures the processors to:
obtain an input feature map and a kernel;
determine a shared operand to be shared in parallelized convolution operations as being either one of a pixel value among pixel values of an input feature map and a weight value among weight values of a kernel based on a shape of the input feature map and dimensions of the kernel; and
perform the parallelized convolution operations using the shared operand, wherein the shared operand is determined as:
being the pixel value of the input feature map in response to a two-dimensional area size of the input feature map being less than or equal to a first threshold value; and
being the weight value of the kernel in response to a two-dimensional area size of the input feature map greater than the first threshold value,
wherein the first threshold value is a reference value used to determine whether the shared operand is the pixel value of the input feature map or the weight value of the kernel.
2. The apparatus of claim 1 , wherein the shared operand, the pixel value of the input feature map, and the weight value of the kernel are of a first layer of a neural network, and
wherein the one or more processors are further configured to:
determine, for a second layer of the neural network, a second shared operand of the second layer to be either one of a pixel value of among pixel values of an input feature map of the second layer and a weight value among weight values of a kernel of the second layer, based on a second shape of the input feature map of the second layer and second dimensions of the kernel of the second layer.
3. The apparatus of claim 1 , wherein the one or more processors are further configured to determine the shared operand further based on either one or both of a percentage of pixels having a zero value within the input feature map and a percentage of weights having a zero value within the kernel.
4. The apparatus of claim 3 , wherein the one or more processors are further configured to determine the shared operand to be the pixel value of the input feature map in response to the percentage of the pixels having a zero value is greater than a third threshold value, and the shared operand to be the weight value of the kernel in response to the percentage of the weights having a zero value is greater than a fourth threshold value.
5. The apparatus of claim 1 , wherein the one or more processors are configured to:
determine the shared operand to be the weight value of the kernel in response to determining that a number of channels of the input feature map is smaller than a second threshold value and the two-dimensional area size of the input feature map is greater than the first threshold value; and
determine the shared operand to be the pixel value of the input feature map in response to determining that the number of the channels of the input feature map is greater than or equal to the second threshold value and the two-dimensional area size of the input feature map is less than or equal to the first threshold value.
6. A processor-implemented neural network method, the method comprising:
obtaining an input feature map and a kernel;
determining a shared operand to be shared in parallelized convolution operations as being either one of a pixel value among pixel values of an input feature map and a weight value among weight values of a kernel based on a shape of the input feature map and dimensions of the kernel; and
performing the parallelized convolution operations using the shared operand,
wherein the shared operand is determined as:
being the pixel value of the input feature map in response to a two-dimensional area size of the input feature map being less than or equal to a first threshold value; and
being the weight value of the kernel in response to a two-dimensional area size of the input feature map greater than the first threshold value,
wherein the first threshold value is a reference value used to determine whether the shared operand is the pixel value of the input feature map or the weight value of the kernel.
7. The method of claim 6 , wherein the shared operand, the pixel value of the input feature map, and the weight value of the kernel are of a first layer of a neural network, and
wherein the method further comprises:
determining, for a second layer of the neural network, a second shared operand of the second layer to be either one of a pixel value of among pixel values of an input feature map of the second layer and a weight value among weight values of a kernel of the second layer, based on a second shape of the input feature map of the second layer and second dimensions of the kernel of the second layer.
8. The method of claim 6 , wherein the determining the shared operand comprises determining the shared operand further based on either one or both of a percentage of pixels having a zero value within the input feature map and a percentage of weights having a zero value within the kernel.
9. The method of claim 8 , wherein the determining the shared operand comprises determining the shared operand to be the pixel value of the input feature map in response to the percentage of the pixels having a zero value is greater than a third threshold value, and the shared operand to be the weight value of the kernel in response to the percentage of the weights having a zero value is greater than a fourth threshold value.
10. The method of claim 6 , wherein the determining the shared operand comprises:
determining the shared operand to be the weight value of the kernel in response to determining that a number of channels of the input feature map is smaller than a second threshold value and the two-dimensional area size of the input feature map is greater than the first threshold value; and
determining the shared operand to be the pixel value of the input feature map in response to determining that the number of the channels of the input feature map is greater than or equal to the second threshold value and the two-dimensional area size of the input feature map is less than or equal to the first threshold value.
11. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of claim 6 .