Method and apparatus for processing image based on neural network

1 . An image processing method comprising:

receiving an input image having a first resolution; and

estimating a preview image of the input image having a second resolution that is lower than the first resolution by using a neural network model,

wherein the neural network model is configured to generate a characteristic vector comprising a vector representation of a pixel-level image characteristic of the input image having the first resolution,

wherein the neural network model is configured to apply the pixel-level image characteristic of the input image having the first resolution to the preview image having the second resolution based on the characteristic vector, and

wherein the pixel-level image characteristic observed from the input image having the first resolution is observed from the preview image having the second resolution.

2 . The method of claim 1 , wherein the pixel-level image characteristic is not observed from the input image having the second resolution when the input image is downscaled to have the second resolution without applying the pixel-level image characteristic of the input image to the preview image, and wherein the pixel-level image characteristic is observed from the preview image according to the application of the pixel-level image characteristic of the input image to the preview image.

3 . The method of claim 1 , wherein the pixel-level image characteristic comprises a degradation characteristic of the input image.

4 . The method of claim 1 , wherein the estimating of the preview image comprises:

estimating the pixel-level image characteristic of the input image based on the input image;

estimating a temporarily restored image having the second resolution by performing image downscaling and image restoration based on the input image; and

applying the pixel-level image characteristic to the temporarily restored image.

5 . The method of claim 4 , wherein the neural network model comprises:

a first model configured to estimate the pixel-level image characteristic of the input image;

a second model configured to estimate the temporarily restored image; and

a third model configured to apply the pixel-level image characteristic to the temporarily restored image.

6 . The method of claim 5 ,

wherein the first model is configured to output the characteristic vector, and

wherein the third model is configured to output the preview image according to an input of the temporarily restored image and the characteristic vector.

7 . The method of claim 5 , wherein the neural network model comprises:

a fourth model configured to estimate the pixel-level image characteristic of the input image, estimate the temporarily restored image, and apply the pixel-level image characteristic to the temporarily restored image.

8 . The method of claim 1 , wherein the estimating of the preview image comprises:

estimating the pixel-level image characteristic of the input image based on the input image;

estimating a temporary image having the second resolution by performing image downscaling based on the input image; and

applying the pixel-level image characteristic to the temporary image.

9 . The method of claim 8 , wherein the neural network model comprises:

a first model configured to estimate the pixel-level image characteristic of the input image;

a fifth model configured to estimate the temporary image, and

a third model configured to apply the pixel-level image characteristic to the temporary image.

10 . The method of claim 1 , further comprising:

generating candidate restored images having the second resolution by performing a restoration task according to a plurality of candidate restoration characteristics on the preview image; and

generating a restored image having the first resolution by performing a restoration task on the input image according to a candidate restoration characteristic selected from the plurality of candidate restoration characteristics.

11 . The method of claim 10 , wherein image characteristics according to the candidate restoration characteristics observed from the candidate restored images having the first resolution when the candidate restored images are upscaled to have the first resolution, are observed from the candidate restored images having the second resolution.

12 . A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .

13 . An image processing apparatus comprising:

a processor; and

a memory configured to store instructions executable by the processor,

wherein, in response to the instructions being executed by the processor, the processor is configured to:

receive an input image having a first resolution;

estimate a preview image of the input image having a second resolution that is lower than the first resolution by using a neural network model,

wherein the neural network model is configured to apply a pixel-level image characteristic of the input image having the first resolution to the preview image having the second resolution,

wherein the pixel-level image characteristic observed from the input image having the first resolution is observed from the preview image having the second resolution, and

wherein, to estimate the preview image, the processor is configured to:

estimate the pixel-level image characteristic of the input image based on the input image;

estimate a temporarily restored image having the second resolution by performing image downscaling and image restoration based on the input image; and

apply the pixel-level image characteristic to the temporarily restored image.

14 . The apparatus of claim 13 , wherein the pixel-level image characteristic is not observed from the input image having the second resolution when the input image is downscaled to have the second resolution without applying the pixel-level image characteristic of the input image to the preview image, and wherein the pixel-level image characteristic is observed from the preview image according to the application of the pixel-level image characteristic of the input image to the preview image.

15 . The apparatus of claim 13 , wherein, to estimate the preview image, the processor is configured to:

estimate the pixel-level image characteristic of the input image based on the input image;

estimate a temporary image having the second resolution by performing image downscaling based on the input image; and

apply the pixel-level image characteristic to the temporary image.

16 . The apparatus of claim 13 , wherein the processor is configured to:

generate candidate restored images having the second resolution by performing a restoration task according to a plurality of candidate restoration characteristics on the preview image; and

generate a restored image having the first resolution by performing a restoration task on the input image according to a candidate restoration characteristic selected from the plurality of candidate restoration characteristics.

17 . The apparatus of claim 16 , wherein image characteristics according to the candidate restoration characteristics observed from the candidate restored images having the first resolution when the candidate restored images are upscaled to have the first resolution are observed from the candidate restored images having the second resolution.

18 . An electronic apparatus comprising:

a camera configured to generate an input image; and

a processor configured to:

receive an input image having a first resolution; and

estimate a preview image of the input image having a second resolution that is lower than the first resolution by using a neural network model,

wherein the neural network model is configured to generate a characteristic vector comprising a vector representation of a pixel-level image characteristic of the input image having the first resolution,

wherein the neural network model is configured to apply the pixel-level image characteristic of the input image having the first resolution to the preview image having the second resolution based on the characteristic vector, and

wherein the pixel-level image characteristic observed from the input image having the first resolution is observed from the preview image having the second resolution.

19 . The electronic apparatus of claim 18 , wherein the pixel-level image characteristic is not observed from the input image having the second resolution when the input image is downscaled to have the second resolution without applying the pixel-level image characteristic of the input image to the preview image, and wherein the pixel-level image characteristic is observed from the preview image according to the application of the pixel-level image characteristic of the input image to the preview image.