Image processing device and method for operating same

1. An image processing apparatus for performing subpixel rendering by using one or more deep neural networks comprising:

a memory storing one or more instructions; and

a processor configured to execute the one or more instructions stored in the memory to;

generate first feature maps by performing convolution operation between a first image and first kernels included in the one or more deep neural networks;

generate second feature maps by upscaling the first feature maps;

generate third feature maps for each of subpixels based on the second feature maps and position information of each of the subpixels arranged in a display;

generate result image for each of the subpixels by performing convolution operation between the third feature maps and second kernels included in the one or more deep neural networks; and

output a second image based on the result image for each of the subpixels.

2. The image processing apparatus of claim 1 , wherein the one or more deep neural networks include a deconvolution layer, and

the processor upscales the first feature maps by performing a deconvolution operation between the third feature maps and third kernels in the deconvolution layer.

3. The image processing apparatus of claim 1 , wherein the subpixels include first to third subpixels, and

the processor generates first sub feature maps for first subpixel by grouping features having a same position relationship with the first subpixel among features included in the second feature maps, generates second sub feature maps for second subpixel by grouping features having a same position relationship with the second subpixel among features included in the second feature maps, and generates third sub feature maps for third subpixel by grouping features having a same position relationship with the third subpixel among features included in the third feature maps.

4. The image processing apparatus of claim 3 , wherein the processor

determines a weight of each of the first sub feature maps based on a proximity of features included in each of the first sub feature maps to the first subpixel,

determines a weight of each of the second sub feature maps based on a proximity of features included in each of the second sub feature maps to the second subpixel,

determines a weight of each of the third sub feature maps based on a proximity of features included in each of the third sub feature maps to the third subpixel, and

generate result image for each of the subpixels by applying weights to each of the first to third sub feature maps.

5. The image processing apparatus of claim 3 , wherein the processor

generates a first result image for the first subpixel by performing a convolution operation between the first sub feature maps and fourth kernels included in the second kernels,

generates a second result image for the second subpixel by performing a convolution operation between the second sub feature maps and fifth kernels included in the second kernels,

generates a third result image including third result values for the third subpixel by performing a convolution operation between the third sub feature maps and fifth kernels, and

wherein the first result image includes output values for the first subpixel, the second result image includes output values for the second subpixel, and the third result image includes output values for the third subpixel.

6. The image processing apparatus of claim 1 , wherein the first image and the second image have a same resolution.

7. A method of operating an image processing apparatus that performs subpixel rendering by using one or more deep neural networks, the method comprising:

generating first feature maps by performing convolution operation between a first image and first kernels included in the one or more deep neural networks;

generating second feature maps by upscaling for the first feature maps;

generating third feature maps for each of subpixels based on the second feature maps and position information of each of the subpixels arranged in a display;

generating result image for each of the subpixels by performing convolution operation between the third feature maps and second kernels included in the one or more deep neural networks; and

outputting a second image based on the result image for each of the subpixels.

8. The method of claim 7 , wherein the one or more deep neural networks include a deconvolution layer, and

the generating of the second feature maps comprises performing a deconvolution operation between the first feature maps and third kernels in the deconvolution layer.

9. The method of claim 7 , wherein the subpixels include first to third subpixels, and

the generating of the second feature maps comprises:

generating first sub feature maps for first subpixel by grouping same position relationship with the first subpixel among features included in the second feature maps;

generating second sub feature maps for second subpixel by grouping features having a same position relationship with the second subpixel among features included in the second feature maps; and

generating third sub feature maps for third subpixel by grouping features having a same position relationship with the third subpixel among features include in the first feature maps.