Image processing apparatus and control method thereof

1. An image processing apparatus comprising:

a communication circuitry configured to communicate with an external device;

a storage configured to store data;

an image processor configured to perform image processing; and

a controller configured to perform an operation, through a neural network, on an image frame contained in an image received by the communication circuitry, to determine a content type of the image based on information according to the operation through the neural network, and to control the image processor based on the determined content type of the image,

wherein the information according to the operation through the neural network comprises a content type and a reliability value of the content type with respect to the image frame, and

wherein the controller is configured to re-perform the operation through the neural network on the image frame or another image frame contained in the image based on the reliability value being less than a predetermined reference value, and determine the content type of the image according to the re-performed operation through the neural network.

2. The image processing apparatus of claim 1 , wherein the controller is configured to obtain a plurality of regions of the image frame based on at least one of a predetermined size and positions, and to perform the operation, through the neural network, on the obtained plurality of regions in a predetermined order.

3. The image processing apparatus of claim 1 , wherein based on all the reliability values being less than the predetermined reference value, the controller is configured to determine the content type of the image based on a highest reliability value among the reliability values.

4. The image processing apparatus of claim 1 , wherein the controller is configured to perform the operation, through the neural network, on a first image frame contained in the image, and based on the reliability value of the first image frame being less than the predetermined reference value, the controller is configured to perform the operation, through the neural network, on a second image frame, which is arranged a predetermined period after the first image frame.

5. The image processing apparatus of claim 1 , wherein the communication circuitry is configured to receive a computer program for the neural network from a server, and the controller is configured to request, from the server, an operation process through the neural network.

6. The image processing apparatus of claim 1 , wherein the neural network comprises a convolution neural network (CNN) generating a feature map that is output by performing convolution on the image frame, and wherein the generated feature map is subsequently input into the neural network.

7. The image processing apparatus of claim 1 , wherein the storage stores a computer program related to the neural network, and the controller is configured to perform the operation through the neural network based on the stored computer program.

8. The image processing apparatus of claim 2 , wherein the image processor is configured to selectively perform at least one of noise processing or frame rate conversion (FRC).

9. The image processing apparatus of claim 5 , wherein the controller is configured to perform the operation, through the neural network, on a first image frame contained in the image and to transmit the first image frame to the server, and after the first image frame is transmitted to the server, the controller is configured to determine the content type of the image by comparing a first reliability value resulting from the operation with a second reliability value received from the server.

10. The image processing apparatus of claim 5 , wherein the controller is configured to request an updated neural network from the server.

11. The image processing apparatus of claim 8 , wherein the controller is configured to determine whether to perform the FRC based on the determined content type of the image.

12. A control method of an image processing apparatus, the control method comprising:

performing an operation, through a neural network, on an image frame contained in an image;

determining a content type of the image based on information according to the operation through the neural network; and

performing image processing based on the determined content type of the image,

wherein the information according to the operation through the neural network comprises a content type and a reliability value of the content type with respect to the image frame,

wherein the performing the operation comprises re-performing the operation, through the neural network, on the image frame or another image frame contained in the image, based on the reliability value being less than a predetermined reference value, and

wherein the determining the content type of the image comprises determining the content type of the image according to the re-performed operation through the neural network.

13. The control method of claim 12 , wherein the re-performing the operation comprises:

obtaining a plurality of regions of the image frame based on at least one of a predetermined size and positions; and

performing the operation, through the neural network, on the acquired plurality of regions in a predetermined order.

14. The control method of claim 12 , wherein the performing the image processing comprises selectively performing at least one of noise processing or frame rate conversion (FRC).

15. The control method of claim 12 , further comprising:

storing a computer program performing the operation through the neural network.

16. The control method of claim 12 , wherein the performing the operation comprises requesting the performing of the operation, through the neural network, to a server.

17. A non-transitory computer-readable recording medium having recorded thereon instructions executable by at least one processor to perform the control method of claim 12 .

18. An image processing apparatus comprising:

an image processor configured to perform image processing on an image; and

a controller configured to perform an operation, through a neural network, on an image frame contained in the image to determine a content type of the image, and to control the image processor to perform the image processing based on the determined content type of the image,

wherein the controller is configured to obtain, as an output of the operation from the neural network, the content type and a reliability value of the content type with respect to the image frame, and

wherein the controller is configured to re-perform the operation through the neural network on the image frame or another image frame contained in the image based on the reliability value being less than a predetermined reference value, and determine the content type of the image according to the re-performed operation through the neural network.

19. The image processing apparatus of claim 18 , wherein:

based on the reliability value being greater than the predetermined reference value, the controller is configured to determine the content type.

20. The image processing apparatus of claim 18 , wherein the controller is configured to crop the image frame and perform the operation, through the neural network, on the cropped image frame.

21. The image processing apparatus of claim 18 , wherein the controller is configured to control the image processor to selectively perform at least one of noise processing or frame rate conversion (FRC) based on the determined content type of the image.

22. The image processing apparatus of claim 18 , wherein based on all the reliability values being less than the predetermined reference value, the controller is configured to determine the content type of the image based on a highest reliability value among the reliability values.