Artificial intelligence encoding and artificial intelligence decoding methods and apparatuses using deep neural network
Provided is an artificial intelligence (AI) encoding apparatus including a memory storing one or more instructions, and a processor configured to execute the one or more instructions stored in the memory to obtain a first image by performing AI down-scaling on an original image through a deep neural network (DNN) for down-scaling, obtain artifact information indicating an artifact region in the first image, perform post-processing to change a pixel value of a pixel in the first image, based on the artifact information, and obtain image data corresponding to a result of encoding of the post-processed first image, and AI data including the artifact information.
1. An electronic device for displaying an image by using an artificial intelligence (AI), the electronic device comprising:
a display; and
one or more processors configured to execute one or more instructions stored in the electronic device to:
receive image data corresponding to an encoding result on a first image;
receive AI data related to AI down-scaling an original image to the first image, the AI data comprising artifact information and a bitrate of the image data being used to select first neural network (NN) setting information, the artifact information comprising a location of an artifact region in the first image;
obtain a second image by decoding the image data;
based on the AI data, select the first NN setting information from a plurality of first NN setting information that is pre-stored in the electronic device;
obtain a third image by performing AI up-scaling on the second image through an up-scaling NN that is set with the selected first NN setting information;
change pixel values of pixels in the third image, based on the artifact information; and
provide the third image of which the pixel values are changed, to the display,
wherein the first image is obtained through a down-scaling NN configured with selected second NN setting information from among a plurality of second NN setting information for the AI down-scaling, and
wherein the plurality of first NN setting information and the plurality of second NN setting information are obtained through joint training of the down-scaling NN and the up-scaling NN.
2. The electronic device of claim 1 , wherein the artifact information comprises an artifact map, and
wherein the one or more processors are further configured to execute the one or more instructions to up-scale the artifact map.
3. The electronic device of claim 1 , wherein the one or more processors are further configured to execute the one or more instructions to change a weight of a filter set for changing the pixel values based on the artifact information, among a plurality of filter sets.
4. The electronic device of claim 1 , wherein the one or more processors further comprise:
a first processor configured to receive the image data and decode the image data to obtain the second image,
a second processor configured to receive the AI data and the second image from the first processor, and output the third image and the artifact information, and
a third processor configured to change the pixel values of pixels in the third image, based on the artifact information.
5. A method of displaying an image by an electronic device configured to use an artificial intelligence (AI), the method comprising:
receiving image data corresponding to an encoding result on a first image;
receiving AI data related to AI down-scaling an original image to the first image, the AI data comprising artifact information and a bitrate of the image data being used to select first neural network (NN) setting information, the artifact information comprising a location of an artifact region in the first image;
obtaining a second image by decoding the image data;
based on the AI data, selecting the first NN setting information from a plurality of first NN setting information that is pre-stored in the electronic device;
obtaining a third image by performing AI up-scaling on the second image, through an up-scaling NN that is set with the selected first NN setting information;
changing pixel values of pixels in the third image, based on the artifact information; and
providing, on a display, the third image of which the pixel values are changed,
wherein the first image is obtained through a down-scaling NN configured with selected second NN setting information from among a plurality of second NN setting information for the AI down-scaling, and
wherein the plurality of first NN setting information and the plurality of second NN setting information are obtained through joint training of the down-scaling NN and the up-scaling NN.
6. A non-transitory computer-readable recording medium having recorded thereon a program which, when executed by the electronic device, performs the method of claim 5 .