Method and apparatus for image decoding and image encoding using AI prediction block
A method of decoding an image, including obtaining a motion vector of a current block; obtaining a preliminary prediction block based on a reference block indicated by the motion vector in a reference image; obtaining a final prediction block for the current block by applying, to a neural network, at least one of a picture order count (POC) map including a POC difference between the reference image and a current image including the current block, the preliminary prediction block, and a quantization error map; and reconstructing the current block based on the final prediction block and a residual block obtained from a bitstream, wherein sample values of the quantization error map are calculated based on a quantization parameter for the reference block.
1. A method of decoding an image, the method comprising:
obtaining a motion vector of a current block;
obtaining a first prediction block based on a reference block in a reference image indicated by the motion vector;
obtaining a second prediction block for the current block by applying, to a neural network, at least one of an enlarged picture order count (POC) map comprising a POC difference between the reference image and a current image comprising the current block, an enlarged first prediction block, or an enlarged quantization error map; and
reconstructing the current block based on the second prediction block,
wherein sample values of a quantization error map are calculated based on a quantization parameter for the reference block,
wherein the at least one of the enlarged POC map, the enlarged first prediction block, or the enlarged quantization error map is obtained by padding at least one of a POC map, the first prediction block, or the quantization error map according to an enlarged distance.
2. The method of claim 1 , wherein the sample values of the quantization error map correspond to a quantization step size or a quantization error value calculated based on the quantization parameter for the reference block.
3. The method of claim 1 , wherein the quantization error map is divided into sub-areas corresponding to lower blocks of the reference block, and
wherein sample values respectively included in the sub-areas of the quantization error map are calculated based on a quantization parameter for a sample at a position in the lower blocks of the reference block.
4. The method of decoding an image of claim 1 , wherein the neural network comprises one or more convolution layers, and
wherein the method further comprises:
determining the enlarged distance based on a number of the one or more convolution layers included in the neural network, a size of a filter kernel used in the one or more convolution layers, and a stride; and
obtaining the enlarged first prediction block comprising neighboring samples corresponding to the enlarged distance from among neighboring samples outside a boundary of the reference block in the reference image and samples of the first prediction block.
5. The method of claim 4 , further comprising obtaining the enlarged quantization error map comprising sample values calculated from a quantization parameter for neighboring samples corresponding to the enlarged distance in the reference image and the quantization parameter for the reference block.
6. The method of claim 4 , wherein, based on a boundary of the reference block corresponding to a boundary of the reference image, neighboring samples corresponding to the enlarged distance are determined from a closest sample available in the reference image.
7. The method of claim 1 , wherein the obtaining of the second prediction block for the current block comprises applying, to the neural network, an enlarged current reconstructed block together with the at least one of the enlarged POC map, the enlarged first prediction block, or the enlarged quantization error map, and
wherein the enlarged current reconstructed block comprises neighboring samples which were reconstructed before the current block in the current image and samples other than samples corresponding to the neighboring samples from among samples of the enlarged first prediction block.
8. The method of claim 1 , further comprising selecting a weight set used to obtain the second prediction block from among a plurality of weight sets, based on at least one of a size of the current block, a prediction direction of the current block, the quantization parameter for the reference block, a layer to which the current image belongs in a hierarchical structure of an image, or information obtained from a bitstream, and
wherein the second prediction block is obtained based on the neural network operating according to the selected weight set.
9. A non-transitory computer-readable recording medium having recorded thereon a program for performing the method of claim 1 , on a computer.
10. A method of encoding an image, the method comprising:
obtaining a motion vector indicating a reference block in a reference image corresponding to a current block;
applying, to a neural network, at least one of an enlarged picture order count (POC) map comprising a POC difference between the reference image and a current image comprising the current block, an enlarged first prediction block obtained based on the reference block, or an enlarged quantization error map to obtain a second prediction block for the current block; and
generating a bitstream by using the second prediction block,
wherein sample values of a quantization error map are calculated based on a quantization parameter for the reference block,
wherein the at least one of the enlarged POC map, the enlarged first prediction block, or the enlarged quantization error map is obtained by padding at least one of a POC map, a first prediction block, or the quantization error map according to an enlarged distance.
11. The method of claim 10 , further comprising changing precision of the obtained motion vector from fractional precision to integer precision,
wherein the reference block indicated by the motion vector having the integer precision is determined as the first prediction block.
12. An image decoding apparatus comprising:
at least one memory configured to store at least one instruction; and
at least one processor configured to execute the at least one instruction to:
obtain a motion vector of a current block,
obtain a first prediction block based on a reference block in a reference image indicated by the motion vector,
obtain a second prediction block for the current block by applying, to a neural network, at least one of an enlarged picture order count (POC) map comprising a POC difference between the reference image and a current image comprising the current block, an enlarged first prediction block, or an enlarged quantization error map, and
reconstruct the current block based on the second prediction block,
wherein sample values of a quantization error map are calculated based on a quantization parameter for the reference block,
wherein the at least one of the enlarged POC map, the enlarged first prediction block, or the enlarged quantization error map is obtained by padding at least one of a POC map, the first prediction block, or the quantization error map according to an enlarged distance.
13. A method of transmitting the bitstream generated by the method of claim 10 .