Image processing apparatus and image processing method
An image processing apparatus and an image processing method where the apparatus includes: a self-encoder configured to perform self-encoding on an input image to generate multiple feature maps; a parameter generator configured to generate multiple convolution kernels for a convolution neural network based on the multiple feature maps; and an outputter configured to generate, by using the convolution neural network, an output result of the input image based on the input image and the multiple convolution kernels. With the image processing apparatus and the image processing method according to the present disclosure, an accuracy of processing an image by using the CNN network can be improved.
1. An image processing apparatus, comprising:
a memory; and
a processor coupled to the memory and configured to:
perform self-encoding on an input image to generate multiple feature maps,
generate, for each of the multiple feature maps, a local feature and a global feature corresponding to a corresponding feature map,
generate, for each of the multiple feature maps, a generated convolution kernel corresponding to the corresponding feature map based on the local feature and the global feature corresponding to the corresponding feature map, and
generate, by using a convolution neural network, an output result for the input image based on the input image and multiple convolution kernels corresponding to the multiple feature maps.
2. The image processing apparatus according to claim 1 , wherein the processor is further configured to:
generate, for each of the multiple feature maps, the local feature corresponding to the corresponding feature map; and
generate, for each of the multiple feature maps, the global feature corresponding to the corresponding feature map based on the multiple local features generated.
3. The image processing apparatus according to claim 2 , wherein the processor is further configured to perform a pooling operation on the corresponding feature map to generate the local feature corresponding to the corresponding feature map.
4. The image processing apparatus according to claim 2 , wherein the processor is further configured to:
connect generated multiple local features; and
input connected multiple local features to a fully-connected network to generate the global feature,
wherein fully-connected networks through which the global features corresponding to different feature maps are generated are different.
5. The image processing apparatus according to claim 1 , wherein the processor is further configured to:
perform nonlinear mapping on a connected local feature and a connected global feature to generate a weight matrix; and
generate a convolution kernel corresponding to the corresponding feature map based on the weight matrix and a reference convolution kernel set,
wherein the reference convolution kernel set includes multiple convolution kernels, and the weight matrix includes a weight for each of the multiple convolution kernels in the reference convolution kernel set.
6. The image processing apparatus according to claim 1 , wherein the processor is further configured to input the input image to multiple second convolvers in sequence to generate an output image,
wherein the multiple convolution kernels are respectively used for the multiple second convolvers.
7. The image processing apparatus according to claim 6 , wherein each of the multiple second convolvers is configured to perform the following operations:
performing a convolution operation on an image input to a second convolver and a convolution kernel for the second convolver;
performing a normalized operation on a convolution result; and
performing a nonlinear mapping on a normalized result.
8. The image processing apparatus according to claim 1 , wherein the input image includes a face image, and the output result includes a feature recognition result for the face image.
9. An image processing method, including:
performing self-encoding on an input image to generate multiple feature maps;
performing, for each of the multiple feature maps, the following operations:
generating a local feature and a global feature corresponding to a corresponding feature map,
generating a convolution kernel corresponding to the corresponding feature map based on the local feature and the global feature corresponding to the corresponding feature map, and
generating, by using a convolution neural network, an output result for the input image based on the input image and multiple convolution kernels corresponding to the multiple feature maps.
10. The image processing method according to claim 9 , wherein the generating the local feature and the global feature corresponding to the corresponding feature map includes:
generating, for each of the multiple feature maps, the local feature corresponding to the corresponding feature map; and
generating, for each of the multiple feature maps, the global feature corresponding to the corresponding feature map based on the generated multiple local features.
11. The image processing method according to claim 10 , wherein a pooling operation is performed on the corresponding feature map to generate the local feature corresponding to the corresponding feature map.
12. The image processing method according to claim 10 , wherein the generating the global feature corresponding to the corresponding feature map based on the generated multiple local features includes:
connecting generated multiple local features; and
inputting connected multiple local features to a fully-connected network to generate the global feature,
wherein fully-connected networks through which the global features corresponding to different feature maps are generated are different.
13. The image processing method according to claim 9 , wherein the generating the convolution kernel corresponding to the corresponding feature map based on the local feature and the global feature includes:
performing nonlinear mapping on a connected local feature and a connected global feature to generate a weight matrix; and
generating a convolution kernel corresponding to the corresponding feature map based on the weight matrix and a reference convolution kernel set,
wherein the reference convolution kernel set includes multiple convolution kernels, and the weight matrix includes a weight for each of the multiple convolution kernels in the reference convolution kernel set.
14. The image processing method according to claim 9 , wherein the generating the output image based on the input image and the multiple convolution kernels includes:
inputting the input image to multiple convolvers in the convolutional neural network in sequence to generate an output image,
wherein the multiple convolution kernels are respectively used for the multiple convolvers.
15. The image processing method according to claim 14 , wherein the inputting the input image to each of the convolvers includes:
performing a convolution operation on an image input to a convolver and a convolution kernel for the convolver;
performing a normalized operation on a convolution result; and
performing a nonlinear mapping on a normalized result.
16. A non-transitory machine readable storage medium on which a program product comprising machine readable instruction codes are carried, wherein the instruction codes, when being read and executed by a computer, cause the computer to perform the method according to claim 9 .
17. The image processing apparatus according to claim 1 , wherein the processor is further configured to input the input image to multiple convolvers in sequence to generate the multiple feature maps.
18. The image processing method according to claim 9 , wherein the performing self-encoding on the input image to generate the multiple feature maps includes:
inputting the input image to multiple convolvers in sequence to generate the multiple feature maps.