Method and apparatus for video coding for improving predicted signals of intra prediction
A video coding method and an apparatus for refining predicted signals in intra prediction are disclosed. The video coding method and apparatus generate refined prediction signals approximating original video signals from predicted signals of intra prediction using a variable and fixed coefficient-based deep learning model to reduce the amount of data for residual signals, which are to be encoded.
1 . A video decoding apparatus comprising:
an entropy decoder configured to decode an intra prediction mode and residual values of a current block from a bitstream, decode a refinement flag, or determine the refinement flag, wherein the refinement flag indicates whether to apply a deep learning-based refinement model at a time of intra prediction of the current block;
an intra predictor configured to generate a prediction block of the current block using the intra prediction mode;
a signal refinement unit configured to generate a refined prediction block from the prediction block using the refinement model when the refinement flag is 1; and
an adder configured to generate a restored block of the current block by adding the residual values to the refined prediction block when the refinement flag is 1 or generate the restored block by adding the residual values to the prediction block when the refinement flag is 0,
wherein the refinement model is implemented using a deep learning model including a fixed coefficient-based network and is pre-trained to generate a refined prediction block approximating an original image of the current block.
2 . The apparatus of claim 1 , wherein the deep learning model further includes a variable coefficient-based network, and the variable coefficient-based network is trained while the fixed coefficient-based network is fixed when the original image of the current block is encoded.
3 . The apparatus of claim 2 , wherein the entropy decoder is configured to decode parameters of the variable coefficient-based network from the bitstream.
4 . The apparatus of claim 1 , wherein the refinement flag is transmitted per block basis, or per picture or slice basis from a video encoding apparatus.
5 . The apparatus of claim 1 , wherein the refinement flag is determined to be 1 when a predetermined prediction mode is used for intra prediction of the current block.
6 . The apparatus of claim 1 , wherein the refinement flag is determined to be 1 when matrix weighted intra prediction is used for intra prediction of the current block.
7 . The apparatus of claim 1 , wherein the refinement flag is determined to be 1 when a plurality of reference lines are used, but reference samples of a row or a column directly adjacent to the current block are not used for intra prediction of the current block.
8 . The apparatus of claim 1 , wherein the refinement flag is determined to be 1 when reference samples are unavailable in an adjacent row or column for intra prediction of the current block.
9 . The apparatus of claim 1 , wherein the refinement flag is determined to be 1 when subblocks partitioned from the current block are used for intra prediction of the current block.
10 . A video encoding method performed by a video encoding apparatus for intra prediction of a current block, the method comprising:
obtaining an intra prediction mode of the current block, obtaining a refinement flag, or determining the refinement flag, wherein the refinement flag indicates whether to apply a deep learning-based refinement model at a time of intra prediction of the current block;
generating a prediction block of the current block using the intra prediction mode; and
generating a residual block of the current block based on the refinement flag,
wherein, when the refinement flag is 1, generating the residual block includes:
generating a refined prediction block from the prediction block using the refinement model; and
generating the residual block by subtracting the refined prediction block from the current block,
wherein, when the refinement flag is 0, generating the residual block includes:
generating the residual block by subtracting the prediction block from the current block,
wherein the refinement model is implemented using a deep learning model including a fixed coefficient-based network and is pre-trained to generate a refined prediction block approximating an original image of the current block.
11 . The method of claim 10 , wherein the deep learning model further includes a variable coefficient-based network, and the variable coefficient-based network is trained while the fixed coefficient-based network is fixed when the original image of the current block is encoded.
12 . The method of claim 11 , further comprising:
encoding parameters of the variable coefficient-based network and transmitting the encoded parameters to a video decoding apparatus.
13 . The method of claim 10 , further comprising:
transmitting the refinement flag to a video decoding apparatus per block basis or transmitting the refinement flag to the video decoding apparatus per picture or slice basis.
14 . The method of claim 10 , wherein determining the refinement flag includes:
determining the refinement flag to be 1 when a predetermined prediction mode is used for intra prediction of the current block.
15 . A method for providing a video decoding device with video data, the method comprising:
encoding the video data into a bitstream; and
transmitting the bitstream to the video decoding device,
wherein encoding the video data comprises:
obtaining an intra prediction mode of a current block, obtaining a refinement flag, or determining the refinement flag, wherein the refinement flag indicates whether to apply a deep learning-based refinement model at a time of intra prediction of the current block;
generating a prediction block of the current block using the intra prediction mode; and
generating a residual block of the current block based on the refinement flag, wherein, when the refinement flag is 1, generating the residual block includes:
generating a refined prediction block from the prediction block using the refinement model; and
generating the residual block by subtracting the refined prediction block from the current block,
wherein, when the refinement flag is 0, generating the residual block includes:
generating the residual block by subtracting the prediction block from the current block,
wherein the refinement model is implemented using a deep learning model including a fixed coefficient-based network and is pre-trained to generate a refined prediction block approximating an original image of the current block.
16 . The method of claim 15 , wherein the deep learning model further includes a variable coefficient-based network, and the variable coefficient-based network is trained while the fixed coefficient-based network is fixed when the original image of the current block is encoded.