Method for encoding/decoding image and device therefor
Provided are an image compressing method including determining a compressed image by performing downsampling using a deep neural network (DNN) on an image; determining a prediction signal by performing prediction based on the compressed image; determining a residual signal based on the compressed image and the prediction signal; and generating a bitstream comprising information about the residual signal, wherein the DNN has a network structure that is predetermined according to training of a downsampling process using information generated in an upsampling process, and an image compressing device for performing the image compressing method. Also, provided are an image reconstructing method of reconstructing a compressed image by using a DNN for upsampling, the compressed image having been compressed by the image compressing method, and an image reconstructing device for performing the image reconstructing method.
1. A method performed by an electronic device for displaying an image, the method comprising:
receiving a bitstream generated by encoding a first image;
decoding the bitstream to obtain a second image;
obtaining a third image upsampled from the second image by using a first deep neural network (DNN) for upsampling, based on upsampling target information; and
providing, on a display of the electronic device, the third image,
wherein the first image is generated by downsampling an original image by using a second DNN for downsampling,
the second DNN is trained based on minimizing a weighted sum of first lossy information, second lossy information and third lossy information,
the first lossy information is based on a first difference between a spatially decreased image and a downsampled image from an original image for training, the spatially decreased image being with respect to the original image for training,
the second lossy information corresponds to structural complexity of the downsampled image, and
the third lossy information is based on a second difference between an upsampled image from the downsampled image and the original image for training, and
wherein the third lossy information is used to train the first DNN.
2. The method of claim 1 , wherein the upsampling target information indicates a conversion degree of resolution of the first image.
3. The method of claim 1 , wherein the upsampling target information is determined based on performance information about the display, compression history information, or a type of the original image.
4. The method of claim 1 , wherein the first DNN is trained based on lossy information obtained by upsampling a downsampled image that is downsampled by the second DNN from the original image for training.
5. The method of claim 1 , wherein the spatially decreased image has a structural characteristic of the original image for training,
wherein the structural characteristic comprises at least one of luminance of the original image for training, contrast of the original image for training, a histogram of the original image for training, an encoding quality, compression history information, or a type of the original image for training.
6. A method for providing an image by a server, the method comprising:
inputting an original image into a second deep neural network (DNN) for downsampling;
obtaining a first image downsampled from the original image by the second DNN;
obtaining image data by encoding the first image, and upsampling target information; and
providing an electronic device with the image data and the upsampling target information,
wherein a second image corresponding to the first image is upsampled by a first DNN for upsampling based on the upsampling target information,
the second DNN is trained based on minimizing a weighted sum of first lossy information, second lossy information and third lossy information,
the first lossy information is based on a first difference between a spatially decreased image and a downsampled image from an original image for training, the spatially decreased image being with respect to the original image for training,
the second lossy information corresponds to structural complexity of the downsampled image, and
the third lossy information is based on a second difference between an upsampled image from the downsampled image and the original image for training, and
wherein the third lossy information is used to train the first DNN.
7. An electronic device for displaying an image, the electronic device comprising:
a display; and
one or more processors, when executing one or more instructions stored in the electronic device, configured to:
receive a bitstream generated by encoding a first image,
decode the bitstream to obtain a second image,
obtain a third image upsampled from the second image by using a first deep neural network (DNN) for upsampling, based on upsampling target information; and
provide, on the display, the third image,
wherein the first image is generated by downsampling an original image by using a second DNN for downsampling,
the second DNN is trained based on minimizing a weighted sum of first lossy information, second lossy information and third lossy information,
the first lossy information is based on a first difference between a spatially decreased image and a downsampled image from an original image for training, the spatially decreased image being with respect to the original image for training,
the second lossy information corresponds to structural complexity of the downsampled image, and
the third lossy information is based on a second difference between an upsampled image from the downsampled image and the original image for training, and
wherein the third lossy information is used to train the first DNN.