Color restoration method and apparatus

1. A color restoration method performed by a color restoration apparatus, the color restoration method comprising:

pre-processing an input image to generate a pre-processed input image;

determining whether color of the pre-processed input image is distorted by using a detected reference image; and

restoring the color of the pre-processed input image in an RGB scale or a grayscale according to whether the color is distorted,

wherein the determining whether the color of the pre-processed input image is distorted comprises:

detecting the reference image to be compared with the pre-processed input image; and

determining whether the color of the pre-processed input image is distorted through color comparison between the pre-processed input image and the reference image,

wherein the detecting the reference image comprises detecting, as the reference image, an image including an object displayed in a single color or an object including a specific color in common among objects displayed in the input image and having an RGB distribution most similar to an RGB distribution of the input image among candidate images based on an RGB distribution on a histogram of the input image, and

wherein the restoring the color of the pre-processed input image in the grayscale comprises:

converting the input image into a grayscale; and

colorizing the grayscale image based on an RGB value of an input image before conversion.

2. The color restoration method according to claim 1 , wherein

the pre-processing the input image comprises noise reduction of the input image, and

the noise reduction includes noise reduction by a low light enhancement model of a deep neural network (DNN).

3. The color restoration method according to claim 1 , wherein

the detecting the reference image comprises detecting an image including an object in the input image and a common object as the reference image.

4. The color restoration method according to claim 1 , wherein

detecting the reference image comprises detecting the reference image based on at least one condition among a light level of the input image, a type of a light source, a location at which the input image is photographed, and a time.

5. The color restoration method according to claim 1 , wherein

the restoring the color of the pre-processed input image comprises restoring the color of the pre-processed input image by using a deep neural network (DNN) trained through deep learning of color restoration for an image photographed in a low light environment.

6. The color restoration method according to claim 5 , wherein the color of the pre-processed input image is restored by using the DNN based on the reference image.

7. The color restoration method according to claim 5 , further comprising:

training the DNN through deep learning of color restoration for an image photographed in a low light environment by using, as input data from a training data set, an image to which at least one among noise reduction, contrast enhancement, super resolution; and brightness enhancement is applied as pre-processing.

8. The color restoration method according to claim 7 , wherein

the deep learning of the color restoration includes on-device learning, which is a type of transfer learning performed on the DNN using personal data other than the training data set.

9. A color restoration method performed by a color restoration apparatus, the color restoration method comprising:

pre-processing an input image to generate a pre-processed input image;

determining whether color of the pre-processed input image is distorted by using a detected reference image; and

restoring the color of the pre-processed input image in an RGB scale or a grayscale according to whether the color is distorted,

wherein the determining whether the color of the pre-processed input image is distorted comprises:

detecting the reference image to be compared with the pre-processed input image; and

determining whether the color of the pre-processed input image is distorted through color comparison between the pre-processed input image and the reference image,

wherein the detecting the reference image comprises detecting, as the reference image, an image having an RGB distribution most similar to an RGB distribution of the input image among candidate images based on an RGB distribution on a histogram of the input image.

10. A color restoration apparatus comprising:

a processor configured to pre-process an input image to generate a pre-processed input image; and

a deep neural network (DNN) model configured to restore distorted color of the pre-processed input image in an RGB scale or a grayscale,

wherein the processor determines whether color of the pre-processed input image is distorted by using a detected reference image, and detects as the reference image, an image including an object displayed in a single color or an object including a specific color in common among objects displayed in the input image, and having an RGB distribution most similar to an RGB distribution of the input image among candidate images based on an RGB distribution on a histogram of the input image, and determines whether the color of the pre-processed input image is distorted through color comparison between the pre-processed input image and the reference image, and

wherein the processor controls the DNN to convert the input image into a grayscale, and to colorize the grayscale image based on an RGB value of an input image before conversion.

11. The color restoration apparatus according to claim 10 , wherein

the processor detects an image including an object in the pre-processed input image and a common object as the reference image, and determines whether the color of the pre-processed input image is distorted through color comparison of the common object.

12. The color restoration apparatus according to claim 10 , wherein

the processor detects the reference image based on at least one condition among a light level of the input image, a type of a light source, a location at which the input image is photographed, and a time.

13. The color restoration apparatus according to claim 10 , wherein

the processor controls the DNN trained through deep learning of color restoration for an image photographed in a low light environment to restore the color of the input image.

14. The color restoration apparatus according to claim 13 , wherein

the processor controls the DNN to restore the color of the input image by using the DNN based on the reference image.