Methods and apparatuses for performing encoding and decoding on image

1. A non-transitory computer-recordable recording medium having stored thereon a video file including artificial intelligence (AI) encoding data,

wherein the video file comprises:

a media data box including image data; and

a metadata box including metadata about the image data,

wherein AI data is included in the metadata box,

wherein the AI encoding data comprises:

the image data including encoding information of a down-scaled image generated by AI down-scaling an original image; and

the AI data about AI up-scaling of the down-scaled image reconstructed according to the image data,

wherein the AI data comprises:

AI target data indicating whether the AI up-scaling is or is not to be performed on at least one frame; and

AI supplementary data about up-scaling deep neural network (DNN) setting information used for performing the AI up-scaling of the at least one frame through an up-scaling deep neural network selected from among a plurality of DNNs related to a plurality of DNN setting information that are pre-stored, when the AI up-scaling is performed on the at least one frame.

2. The non-transitory computer-readable recording medium of claim 1 , wherein the AI target data comprises at least one of:

video AI target data indicating whether the AI up-scaling is to be performed on a plurality of frames included in the image data;

video segment AI target data indicating whether the AI up-scaling is to be performed on a plurality of frames included in a video segment;

frame group AI target data indicating whether the AI up-scaling is to be performed on a plurality of frames included in a frame group; or

frame AI target data indicating whether the AI up-scaling is to be performed on a current frame.

3. The non-transitory computer-readable recording medium of claim 1 , wherein the AI supplementary data comprises at least one of:

video AI supplementary data about at least one piece of up-scaling DNN setting information used to perform the AI up-scaling of all of a plurality of frames included in the image data;

video segment AI supplementary data about at least one piece of up-scaling DNN setting information used to perform the AI up-scaling of the plurality of frames included in a video segment;

frame group AI supplementary data about at least one piece of up-scaling DNN setting information used to perform the AI up-scaling of the plurality of frames included in a frame group; or

frame AI supplementary data about up-scaling DNN setting information used to perform the AI up-scaling of a current frame.

4. The non-transitory computer-readable recording medium of claim 3 , wherein:

the AI data comprises at least one of:

video segment AI supplementary data dependency information indicating whether video segment AI supplementary data is the same between a current video segment and a consecutive previous video segment,

frame group AI supplementary data dependency information indicating whether frame group AI supplementary data is the same between a current frame group and a consecutive previous frame group, or

frame AI supplementary data dependency information indicating whether frame AI supplementary data is the same between a current frame and a consecutive previous frame,

when the video segment AI supplementary data dependency information indicates that the AI segment AI supplementary data is the same between the current video segment and the consecutive previous video segment, the video segment AI supplementary data about the current video segment is omitted from the AI data,

when the frame group AI supplementary data dependency information indicates that the frame group AI supplementary data is the same between the current frame group and the consecutive previous frame group, the frame group AI supplementary data about the current frame group is omitted from the AI data, and

when the frame AI supplementary data dependency information indicates that the frame AI supplementary data is the same between the current frame and the consecutive previous frame, the frame AI supplementary data about the current frame is omitted from the AI data.

5. The non-transitory computer-readable recording medium of claim 1 , wherein:

the metadata box comprises synchronization data about synchronization of the image data and the AI data, and

the AI data is configured to be decoded according to a reproduction order or decoding order of a plurality of frames of the image data, based on the synchronization data.

6. The non-transitory computer-readable recording medium of claim 1 , wherein:

the AI data is embedded in the image data,

video AI data about the AI up-scaling of the image data is located in the image data to be decoded together with a video encoding parameter performed on all of a plurality of frames of the image data,

video segment AI data about the AI up-scaling of a current video segment including a current frame is located in the image data to be decoded together with a video segment encoding parameter on to the plurality of frames of the current video segment,

frame group AI data about the AI up-scaling of a current frame group including the current frame is located in the image data to be decoded together with a frame group parameter performed on the plurality of frames of the current frame group, and

frame AI data about the AI up-scaling of the current frame is located in the image data to be decoded together with encoding information of the current frame included in the plurality of frames.

7. The non-transitory computer-readable recording medium of claim 6 , wherein the image data comprises:

a video header including a video encoding parameter of the image data;

a video segment header including a video segment encoding parameter of the current video segment;

a frame group header including a frame group encoding parameter of the current frame group; and

a frame header including encoding information of the current frame,

the video AI data is included in the video header or is located immediately before or after the video header,

the video segment AI data is included in the video segment header or is located immediately before or after the video segment header,

the frame group AI data is included in the frame group header or is located immediately before or after the frame group header, and

the frame AI data is included in the frame header or is located immediately before or after the frame header.

8. The non-transitory computer-readable recording medium of claim 1 , wherein:

the video file comprises:

a metadata box including metadata about the image data; and

a plurality of video segment data boxes in which the AI encoding data is split according to certain times,

wherein each of the plurality of video segment data boxes comprises:

a segment media data box including video segment data; and

a segment metadata box including metadata about the video segment data, and

the AI data comprises video AI data used to perform the AI up-scaling of all frames of the image data and video segment AI data used to perform the AI up-scaling of all frames of a video segment, and

wherein the video AI data is included in the metadata box and the video segment AI data is included in the segment metadata box.

9. The non-transitory computer-readable recording medium of claim 1 , wherein the AI supplementary data comprises at least one of:

channel information indicating a color channel to which the AI up-scaling is performed;

target bitrate information indicating a bitrate of the downscaled image; or

resolution information related to resolution of the AI up-scaled image or the original image.

10. A method of displaying an image by an electronic device configured to use an artificial intelligence (AI), the method comprising:

receiving a video file including AI encoding data that includes image data and AI data about AI up-scaling of the image data which includes AI target data indicating whether the AI up-scaling is or is not to be performed on at least one frame;

obtaining the AI data of the AI encoding data from a metadata box of the video file and obtaining the image data of the AI encoding data from a media data box of the video file;

reconstructing a down-scaled image of at least one frame by decoding the image data;

obtaining up-scaling deep neural network (DNN) setting information of the at least one frame used to perform the AI up-scaling of at least one frame through an up-scaling deep neural network selected from among a plurality of up-scaling DNNs related to a plurality of up-scaling DNN setting information that are pre-stored, when the AI up-scaling is performed on the at least one frames from the AI data; and

generating a AI up-scaled image corresponding to the down-scaled image by performing the AI up-scaling of the down-scaled image through the selected up-scaling deep neural network; and

providing on a display of the electronic device, the AI up-scaled image.

11. The method of claim 10 , wherein the AI data comprises at least one of frame AI data about the AI up-scaling of a current frame included in the at least one frame, frame group AI data about the AI up-scaling of a frame group including the at least one frame, video segment AI data about the AI up-scaling of a video segment including the at least one frame, or video AI data about the AI up-scaling of a video including the at least one frame.

12. The method of claim 10 , wherein

the AI data further comprises:

AI supplementary data about up-scaling DNN setting information used to AI up-scale the at least one frame,

wherein the obtaining of the up-scaling DNN setting information comprises:

identifying whether the AI up-scaling is performed won the at least one frame, according to the AI target data; and

obtaining at least one piece of up-scaling DNN setting information used to perform the AI up-scaling on the at least one frame, according to the AI supplementary data, when it is identified that the AI up-scaling is performed on the at least one frame.

13. The method of claim 10 , wherein:

the AI data comprises at least one of target bitrate information indicating a bitrate of the down-scaled image or resolution information related to resolution of the AI up-scaled image or the original image, and

the obtaining of the up-scaling DNN setting information comprises obtaining two or more pieces of up-scaling DNN setting information of the at least one frame from a plurality of pieces of default up-scaling DNN setting information, according to at least one of the target bitrate information or the resolution information.

14. The method of claim 10 , wherein:

the metadata box of the video file comprises synchronization data about synchronization of the image data and the AI data, and

the obtaining of the AI data comprises obtaining the AI data corresponding to the at least one frame according to an encoding order or reproduction order of the image data and the AI data indicated by the synchronization data.

15. An electronic device for displaying an image by using an artificial intelligence (AI), the electronic device comprising:

a display; and

one or more processors configured to execute the one or more instructions stored in the electronic device to:

receive a video file including AI encoding data that includes image data and AI data about AI up-scaling of the image data which includes AI target data indicating whether the AI up-scaling is or is not to be performed on at least one frame,

obtain the AI data of the AI encoding data from a metadata box of the video file and obtain the image data of the AI encoding data from a media data box of the video file,

reconstruct a down-scaled image of at least one frame by decoding the image data,

obtain up-scaling deep neural network (DNN) setting information of the at least one frame used to perform the AI up-scaling on at least one frame through an up-scaling deep neural network selected from among a plurality of DNNs related to a plurality of up-scaling DNN setting information that are pre-stored, when the AI up-scaling is performed on the at least one frames from the AI data,

generate a AI up-scaled image corresponding to the down-scaled image by performing the AI up-scaling of the down-scaled image through the selected up-scaling deep neural network, and

provide on the display of the electronic device, the AI up-scaled image.