Image processing apparatus and image processing method thereof
An image processing apparatus applies an image to a first learning network model to optimize the edges of the image, applies the image to a second learning network model to optimize the texture of the image, and applies a first weight to the first image and a second weight to the second image based on information on the edge areas and the texture areas of the image to acquire an output image.
1 . An image processing apparatus comprising:
a memory storing at least one instruction; and
a processor configured to execute the at least one instruction stored in the memory to:
obtain a first image in which a first area is processed and a first image characteristic is enhanced in an input image by applying the input image to a first neural network model,
obtain a second image in which a second area is processed and a second image characteristic is enhanced in the input image by applying the input image to a second neural network model,
obtain a first weight corresponding to the first image and a second weight corresponding to the second image by applying the input image to a third neural network model, and
provide an output image by mixing the first and second images according to the first weight and the second weight,
wherein each of the first neural network model and the second neural network model is trained to enhance different characteristics of an image.
2 . The image processing apparatus of claim 1 , wherein the processor is further configured to identify a plurality of objects included in the input image,
wherein the first area corresponds to a first object among the plurality of objects, and
wherein the second area corresponds to a second object among the plurality of objects.
3 . The image processing apparatus of claim 1 , wherein the processor is further configured to:
identify the first area included in the input image,
obtain the first image in which the first image characteristic of the first area is enhanced in the input image by using the first neural network model,
identify the second area included in the input image,
obtain the second image in which the second image characteristic of the second area is enhanced in the input image by using the second neural network model, and
provide the output image by mixing the first image and the second image.
4 . The image processing apparatus of claim 3 , wherein the first image characteristic of the first area corresponds to a characteristic of a graphic image, and
wherein the second image characteristic of the second area corresponds to a characteristic of a natural image.
5 . The image processing apparatus of claim 1 ,
wherein the first weight corresponds to the first area by,
wherein the second weight corresponds to the second area, and
wherein the processor is further configured to:
apply the first weight to the first image,
apply the second weight to the second image, and
mix the first image to which the first weight is applied and the second image to which the second weight is applied to provide the output image.
6 . The image processing apparatus of claim 1 , wherein the processor is further configured to:
classify each of a plurality of image blocks constituting the input image into either a graphic image or a natural image,
obtain the first image in which the first image characteristic of an image block classified as the graphic image from among the plurality of image blocks is enhanced as the first image by using the first neural network model, and
obtain the second image in which the second image characteristic of an image block classified as the natural image from among the plurality of image blocks is enhanced as the second image by using the second neural network model.
7 . The image processing apparatus of claim 6 , wherein the graphic image includes any one of an illustration image, a computer graphic image, or an animation image, and
wherein the natural image includes any one of a landscape image or a human image.
8 . The image processing apparatus of claim 6 , wherein the image block classified as the graphic image among the plurality of image blocks corresponds to the first area in the input image, and the image block classified as the natural image among the plurality of image blocks corresponds to the second area in the input image.
9 . The image processing apparatus of claim 1 , wherein the first neural network model and the second neural network model are different types of neural network models.
10 . The image processing apparatus of claim 1 , wherein the first neural network model is one of a deep learning model for enhancing the first image using a plurality of layers or a machine learning model trained to enhance the first image characteristic using a plurality of pre-learned filters, and
wherein the second neural network model is one of a deep learning model for enhancing the second image characteristic using a plurality of layers or a machine learning model trained to enhance the second image characteristic using a plurality of pre-learned filters.
11 . A method of controlling an image processing apparatus, the method comprising:
obtaining a first image in which a first area is processed and a first image characteristic is enhanced in an input image by applying the input image to a first neural network model;
obtaining a second image in which a second area is processed and a second image characteristic is enhanced in the input image by applying the input image to a second neural network model;
obtaining a first weight corresponding to the first image and a second weight corresponding to the second image by applying the input image to a third neural network model, and
providing an output image by mixing the first and second images according to the first weight and the second weight,
wherein each of the first neural network model and the second neural network model is trained to enhance different characteristics of an image.
12 . The method of claim 11 , wherein the method further comprises:
identifying a plurality of objects included in the input image,
wherein the first area corresponds to a first object among the plurality of objects, and
wherein the second area corresponds to a second object among the plurality of objects.
13 . The method of claim 11 , wherein the method further comprises:
identifying the first area included in the input image;
identifying the second area included in the input image,
wherein the obtaining the first image comprises obtaining the first image in which a first characteristic of the first area is enhanced in the input image by using the first neural network model,
wherein the obtaining the second image comprises obtaining the second image in which a second characteristic of the second area is enhanced in the input image by using the second neural network model, and
wherein the providing the output image comprises providing the output image by mixing the first image and the second image.
14 . The method of claim 13 , wherein the first characteristic of the first area corresponds to a characteristic of a graphic image, and
wherein the second characteristic of the second area corresponds to a characteristic of a natural image.
15 . The method of claim 11 ,
wherein the first weight corresponds to the first area,
wherein the second weight corresponds to the second area,
wherein the method further comprises:
applying the first weight to the first image;
applying the second weight to the second image; and
mixing the first image to which the first weight is applied and the second image to which the second weight is applied to provide the output image.
16 . The method of claim 11 , wherein the method further comprises:
classifying each of a plurality of image blocks constituting the input image into either a graphic image or a natural image,
wherein the obtaining comprises obtaining the first image in which the first image characteristic of an image block classified as the graphic image from among the plurality of image blocks is enhanced as the first image by using the first neural network model, and
wherein the obtaining comprises obtaining the second image in which the second image characteristic of an image block classified as the natural image from among the plurality of image blocks is enhanced as the second image by using the second neural network model.
17 . The method of claim 16 , wherein the graphic image includes any one of an illustration image, a computer graphic image, or an animation image, and
wherein the natural image includes any one of a landscape image or a human image.
18 . The method of claim 16 , wherein the image block classified as the graphic image among the plurality of image blocks corresponds to the first area in the input image, and the image block classified as the natural image among the plurality of image blocks corresponds to the second area in the input image.
19 . The method of claim 11 , wherein the first neural network model and the second neural network model are different types of neural network models.
20 . The method of claim 11 , wherein the first neural network model is one of a deep learning model for enhancing the first image characteristic using a plurality of layers or a machine learning model trained to enhance the first image characteristic using a plurality of pre-learned filters, and
wherein the second neural network model is one of a deep learning model for enhancing the second image characteristic using a plurality of layers or a machine learning model trained to enhance the second image characteristic using a plurality of pre-learned filters.