Electronic apparatus, system and controlling method thereof

1. An electronic apparatus comprising:

a processor configured to:

obtain quality information of an image using a trained second artificial intelligence (AI) model, the quality information including resolution information and bitrate information,

control downscaling of the image using a trained first AI model based on the resolution information included in the obtained quality information, and

encode the downscaled image based on the bitrate information included in the obtained quality information to generate a frame including the encoded image and the resolution and bitrate information,

wherein the second AI model is trained using feature information of the image obtained from the first AI model.

2. The electronic apparatus of claim 1 , wherein the quality information of the image comprises a plurality of quality indicators corresponding to different combinations of resolution information and bitrate information related to the image,

wherein the processor is configured to:

determine a downscaling ratio of the image and a bitrate of the downscaled image based on resolution information and bitrate information corresponding to any of the plurality of quality indicators, and

control downscaling of the image based on the determined downscaling ratio, and

control encoding of the downscaled image based on the determined bitrate.

3. The electronic apparatus of claim 1 , further comprising:

a communication interface comprising communication circuitry,

wherein the processor is configured to obtain state information of a network to which the communication interface is connected, and to control downscaling of the image based on the state information of the network and the resolution information.

4. The electronic apparatus of claim 3 , wherein the quality information of the image comprises a plurality of quality indicators corresponding to different combinations of information on a plurality of resolutions and information on a plurality of bitrates related to the image,

wherein the processor is configured to:

determine at least one first quality indicator from among the plurality of quality indicators based on the state information of network,

determine a second quality indicator from among the at least one first quality indicator based on a target quality indicator,

control downscaling of the image based on resolution information corresponding to the second quality indicator and control encoding of the downscaled image based on information on the bitrate corresponding to the second quality indicator.

5. The electronic apparatus of claim 1 , wherein the second AI model comprises at least one neural network layer, and

wherein a parameter of the at least one neural network layer is trained jointly with a parameter of at least some layers included in the first AI model.

6. The electronic apparatus of claim 1 , wherein the processor is configured to control downscaling of a current frame based on quality information obtained based on the current frame of the image, or to control downscaling of the current frame based on quality information predicted based on at least one previous frame of the image.

7. The electronic apparatus of claim 6 , wherein the second AI model comprises at least one recurrent neural networks (RNN) layer trained to predict quality information of the current frame based on feature information corresponding to at least one previous frame.

8. The electronic apparatus of claim 1 , wherein the quality information of the image comprises a plurality of quality indicators corresponding to different combinations of information on a plurality of resolutions and information on a plurality of bitrates related to the image, and

wherein the second AI model is trained based on supervised training with a difference between a plurality of quality indicators corresponding to different combinations of information on a plurality of resolutions and information on a plurality of bitrates related to a training image and a plurality of quality indicators output by inputting the training image into the second AI model as a loss function.

9. The electronic apparatus of claim 1 , wherein the processor is configured to:

determine a downscaling ratio of the image and a bitrate of the downscaled image based on the quality information of the image,

downscale the image based on the determined downscaling ratio and encode the downscaled image based on the determined bitrate, and

generate a packet based on the encoded image and transmit the generated packet,

wherein the resolution information and the bitrate information are included in a header of the packet.

10. A method for controlling an electronic apparatus, the method comprising:

obtaining quality information of an image using a trained second artificial intelligence (AI) model, the quality information including resolution information and bitrate information;

downscaling the image using a trained first AI model based on the resolution information included in the obtained quality information; and

encoding the downscaled image based on the bitrate information included in the obtained quality information to generate a frame including the encoded image and the resolution and bitrate information,

wherein the second AI model is trained using feature information of the image obtained from the first AI model.

11. The method of claim 10 , wherein the quality information of the image comprises a plurality of quality indicators corresponding to different combinations of resolution information and bitrate information related to the image,

wherein the downscaling the image comprises determining a downscaling ratio of the image based on any one of the plurality of quality indicators and controlling downscaling of the image based on the determined downscaling ratio, and

wherein the encoding the downscaled image comprises determining a bitrate of the downscaled image based on the any one of the quality indicators and controlling encoding of the downscaled image based on the determined bitrate.

12. The method of claim 10 , further comprising:

obtaining state information of a network,

wherein the downscaling the image comprises controlling downscaling of the image based on the state information of the network and the resolution information, and

wherein the encoding the downscaled image comprises controlling encoding of the downscaled image based on the state information of the network and the bitrate information.

13. The method of claim 12 , wherein the quality information of the image comprises a plurality of quality indicators corresponding to different combinations of information on a plurality of resolutions and information on a plurality of bitrates related to the image, and the method further comprises:

determining at least one first quality indicator from among the plurality of quality indicators based on the state information of network and determining a second quality indicator from among the at least one first quality indicator based on a target quality indicator,

wherein the downscaling the image comprises controlling downscaling of the image based on resolution information corresponding to the determined second quality indicator, and

wherein the encoding the downscaled image comprises controlling encoding of the downscaled image based on information on the bitrate corresponding to the determined second quality indicator.

14. The method of claim 10 , wherein the second AI model comprises at least one neural network layer, and

wherein a parameter of the at least one neural network layer is trained jointly with parameter of at least some layers included in the first AI model.

15. The method of claim 10 , wherein the downscaling of the image comprises controlling downscaling of the current frame based on quality information obtained based on a current frame of the image, or controlling downscaling of the current frame based on quality information predicted based on at least one previous frame of the image.

16. An electronic apparatus comprising:

a communication interface comprising communication circuitry; and

a processor configured to:

downscale an image; and

encode the downscaled image,

wherein the processor is further configured to:

control downscaling of the image based on quality information of the image,

downscale the image using a trained first artificial intelligence (AI) model,

control the downscaling of the image based on the quality information of the image obtained using a trained second AI model and state information of a network to which the communication interface is connected,

encode the downscaled image based on the quality information of the image obtained using the second AI model, and

wherein the second AI model is trained using feature information of the image obtained from the first AI model to obtain the quality information of the image,

wherein the quality information of the image comprises a plurality of quality indicators corresponding to different combinations of resolution information and bitrate information related to the image,

wherein the processor is further configured to:

determine a downscaling ratio of the image and a bitrate of the downscaled image based on resolution information and bitrate information corresponding to any of the plurality of quality indicators, and

control downscaling of the image based on the determined downscaling ratio, and control encoding of the downscaled image based on the determined bitrate.