Method and device for encoding or decoding image
Provided is in-loop filtering technology using a trained deep neural network (DNN) filter model. An image decoding method according to an embodiment includes receiving a bitstream of an encoded image, generating reconstructed data by reconstructing the encoded image, obtaining information about a content type of the encoded image from the bitstream, determining a deep neural network (DNN) filter model trained to perform in-loop filtering by using at least one computer, based on the information about the content type, and performing the in-loop filtering by applying the reconstructed data to the determined DNN filter model.
1. An image decoding method comprising:
receiving a bitstream of an encoded image;
generating reconstructed data by reconstructing the encoded image;
obtaining, from the bitstream, information about a content type of the encoded image and information about a quantization parameter (QP) of the encoded image;
from among a first plurality of deep neural network (DNN) filter model candidates trained to perform in-loop filtering, selecting a second plurality of DNN filter model candidates based on one of the information about the content type and the information about the QP;
from among the second plurality of DNN filter model candidates, selecting a DNN filter model based on another one of the information about the content type and the information about the QP; and
performing the in-loop filtering by applying the reconstructed data to the selected DNN filter model,
wherein the information about the content type comprises information indicating a pixel complexity and a degree of motion of the encoded image.
2. The image decoding method of claim 1 , wherein the in-loop filtering comprises at least one operation from among deblocking filtering, sample adaptive offset, and adaptive loop filtering.
3. The image decoding method of claim 1 , wherein the DNN filter model is a network model trained to compensate for a quantization error of the reconstructed data according to an operation based on a weight of each of a plurality of network nodes constituting the DNN filter model and a connection relationship between the plurality of network nodes.
4. The image decoding method of claim 1 , wherein the determining of the DNN filter model comprises determining the DNN filter model corresponding to the content type of the encoded image from among the second plurality of DNN filter model candidates, based on the information about the content type.
5. The image decoding method of claim 4 , wherein each of the first plurality of DNN filter model candidates is trained to perform the in-loop filtering on a preset content type.
6. The image decoding method of claim 1 , wherein the determining of the DNN filter model further comprises determining the DNN filter model corresponding to a compression strength of the encoded image from among the second plurality of DNN filter model candidates.
7. The image decoding method of claim 1 , wherein the performing of the in-loop filtering comprises performing the in-loop filtering by applying the reconstructed data and one or more reference images stored in a reconstructed picture buffer to the selected DNN filter model.
8. The image decoding method of claim 1 , wherein the in-loop filtering is performed based on a convolutational neural network (CNN) learning model.
9. An image decoding apparatus comprising:
a receiver configured to receive a bitstream of an encoded image; and
a decoder configured to:
generate reconstructed data by reconstructing the encoded image;
obtain, from the bitstream, information about a content type of the encoded image and information about a quantization parameter (QP) of the encoded image;
from among a first plurality of deep neural network (DNN) filter model candidates trained to perform in-loop filtering, selecting a second plurality of DNN filter model candidates based on one of the information about the content type and the information about the QP;
from among the second plurality of DNN filter model candidates, selecting a DNN filter model based on another one of the information about the content type and the information about the QP; and
performing the in-loop filtering by applying the reconstructed data to the selected DNN filter model,
wherein the information about the content type comprises information indicating a pixel complexity and a degree of motion of the encoded image.
10. An image encoding method comprising:
determining a content type of an input image and a quantization parameter (QP) of the input image;
from among a first plurality of deep neural network (DNN) filter model candidates trained to perform in-loop filtering, selecting a second plurality of DNN filter model candidates based on one of the content type and the QP:
from among the second plurality of DNN filter model candidates, selecting a DNN filter model based on another one of the content type and the QP;
generating the in-loop filtered data by applying, to the selected DNN filter model, reconstructed data of the input image reconstructed from encoded residual data;
generating prediction data by predicting the input image based on the in-loop filtered data and generating residual data by using the input image and the prediction data;
generating a bitstream by encoding information about the content type and the residual data; and
transmitting the bitstream,
wherein the information about the content type comprises information indicating a pixel complexity and a degree of motion of the input image.
11. The image encoding method of claim 10 , wherein the in-loop filtering comprises at least one operation from among deblocking filtering, sample adaptive offset, and adaptive loop filtering.
12. The image encoding method of claim 10 , wherein the DNN filter model is a network model trained to compensate for a quantization error of the reconstructed data according to an operation based on a weight of each of a plurality of network nodes constituting the DNN filter model and a connection relationship between the plurality of network nodes.
13. The image encoding method of claim 10 , wherein the determining of the DNN filter model comprises determining the DNN filter model corresponding to the content type of the input image from among the second plurality of DNN filter model candidates, based on the information about the content type.
14. The image decoding method of claim 1 , wherein the determining of the DNN filter model comprises selecting the DNN filter model, based on the information indicating the pixel complexity and the degree of the motion of the encoded image, from among the second plurality of DNN filter model candidates trained to perform the in-loop filtering.
15. The image decoding method of claim 1 , wherein the second plurality of DNN filter model candidates is selected based on the information about the QP, and
wherein the selected DNN filter model is selected based on the information about the content type.
16. The image decoding method of claim 1 , wherein the second plurality of DNN filter model candidates is selected based on the information about the content type, and
wherein the selected DNN filter model is selected based on the information about the QP.