Image processing apparatus and control method thereof

1. An image processing apparatus comprising:

communication circuitry;

a storage; and

at least one processor configured to control the image processing apparatus to: perform object recognition for recognizing a plurality of objects in first image data stored in the storage, obtain a first score through at least one neural network that is trained using data indicative of evaluation for aesthetics for each of the recognized plurality of objects, merge at least two objects among the plurality of objects based on the obtained first scores and a proximity of the plurality of objects, obtain a second score for the merged objects through the at least one neural network, generate a second image data using the merged objects in response to the second score being higher than a predetermined reference value, and perform image processing based on the second image data.

2. The image processing apparatus of claim 1 , wherein the at least one processor is configured to control the image processing apparatus to: generate a window in the first image data, and perform the object recognition based on the generated window,

wherein the window is generated based on a predetermined number and size, or generated based on a size or an aspect ratio of the image data on which the object recognition is to be performed.

3. The image processing apparatus of claim 2 , wherein the at least one processor is configured to control the image processing apparatus to: input the generated window to the at least one neural network, and select a window comprising each of the plurality of objects based on a reliability and a type of the object obtained from the at least one neural network.

4. The image processing apparatus of claim 3 , wherein

the score comprises an aesthetic score of an object contained in the selected window, and

the at least one processor is configured to control the image processing apparatus to: extract a feature from the at least one neural work and obtain the aesthetic score based on the extracted feature.

5. The image processing apparatus of claim 4 , wherein

the feature comprises at least one of color histogram features, text features, symmetry, rules of thirds features, or high level semantic features.

6. The image processing apparatus of claim 4 , wherein

the communication circuitry is configured to communicate with a server configured to perform operation processing through the at least one neural network,

wherein the at least one neural network is trained based on labeling contained in data crawling from the server.

7. The image processing apparatus of claim 4 , wherein

the at least one processor is configured to control the image processing apparatus to: determine the proximity based on a distance between the selected windows and merge the plurality of windows based on the obtained score and the proximity.

8. The image processing apparatus of claim 7 , wherein

the at least one is configured to control the image processing apparatus to: input the merged window into the at least one neural network, and compare the obtained second score with the predetermined reference value.

9. The image processing apparatus of claim 8 , wherein

based on the comparison result, the at least one processor is configured to control the image processing apparatus to generate the second image data comprising the merged window.

10. The image processing apparatus of claim 1 , wherein

the at least one processor is configured to control the image processing apparatus to perform image processing on the second image data to adjust a resolution based on the at least one neural network.

11. The image processing apparatus of claim 1 , wherein

the at least one processor is configured to control the image processing apparatus to: classify a genre of the second image data, and perform image processing to transfer the style of the second image data based on the classified genre.

12. The image processing apparatus of claim 1 , further comprising:

a sensor configured to collect data about a surrounding environment of the image processing apparatus,

wherein the at least one processor is configured to control the image processing apparatus to perform image processing to adjust brightness or contrast of the second image based on the collected data.

13. The image processing apparatus of claim 4 , wherein

the at least one processor is configured to control the image processing apparatus to: store a first feature extracted from the at least one neural network based on the selected window, perform the neural network processing on the image processed second image, and extract a second feature from the at least one neural network.

14. The image processing apparatus of claim 13 , wherein

the at least one processor is configured to control the image processing apparatus to classify a mood of the second image.

15. The image processing apparatus of claim 14 , further comprising:

a display configured to output the image processed second image data; and a sound outputter comprising sound output circuitry configured to reproduce a sound source,

wherein the at least one processor is configured to control the image processing apparatus to recommend a playlist based on the classified mood, control the sound outputter to output the sound source contained in the playlist, and control the display to output the image-processed second image data.

16. A method of controlling an image processing apparatus comprising:

performing object recognition for recognizing a plurality of objects in first image data;

obtaining a first score through at least one neural network that is trained using data indicative of evaluation for aesthetics for each of the recognized plurality of objects;

merging at least two objects among the plurality of objects based on the obtained first scores and a proximity of the plurality of objects,

obtaining a second score for the merged objects through the at least one neural network,

generating a second image data using the merged objects in response to the second score being higher than a predetermined reference value;

and

performing image processing on the second image data based on features extracted from the neural network.

17. The method of claim 16 , wherein

the performing of the object recognition comprises:

generating a window in the first image data,

performing operation processing on the generated window through the neural network; and

selecting a window comprising each of the plurality of objects based on a reliability and a type of the object through the neural network processing.

18. The method of claim 16 , wherein

the performing image processing on the second image comprises:

performing operation processing on the second image data through the neural network;

adjusting a resolution based on the result from the operation processing through the neural network;

classifying a genre of the second image; and

transferring a style of the second image data based on the classified genre.

19. The method of claim 17 , further comprising:

storing a first feature extracted in the operation processing through the neural network based on the selected window;

performing the operation processing on the image processed second image data through the neural network;

re-performing the operation processing on the second feature and the first feature extracted from the neural network processing through the neural network; and

outputting a sound source based on a playlist received based on a mood of the second image data inferred in the operation processing.

20. A method of controlling an image processing apparatus comprising:

performing object recognition for recognizing a plurality of objects in first image data;

obtaining scores inferred through operation processing through a neural network for the recognized plurality of objects;

generating second image data based on at least the obtained score and a proximity of the plurality of objects; and

performing image processing on the second image data based on features extracted from the neural network;

wherein the performing of the object recognition comprises: generating a window in the first image data, performing operation processing on the generated window through the neural network; and selecting a window comprising each of the plurality of objects based on a reliability and a type of the object;

wherein the generating comprises: determining the proximity based on a distance between the selected windows; merging the plurality of windows based on the score and the proximity; performing operation processing on the merged window through the neural network; comparing the score with a predetermined reference value; and generating a second image comprising the merged window based on a result of the comparing.