Method and apparatus for video coding using matrix based cross-component prediction
View Patent ↗A method and an apparatus for video coding using a matrix-based cross-component prediction are disclosed. The video coding method and apparatus predict a chroma component of a current block, by using a deep learning-based matrix operation, from a chroma component spatially adjacent to a chroma block of the current block and from a luma component spatially adjacent to a luma block corresponding to the chroma block.
1 . A method performed by a video decoding apparatus for predicting a chroma component of a current block using a luma component, the method comprising:
obtaining reference pixels that include chroma reference pixels spatially adjacent to a chroma block of the current block and include luma reference pixels adjacent to a luma block corresponding to the chroma block;
generating an input block to be inputted into a neural network-based estimating model by using the reference pixels, wherein the input block is formed as a one-dimensional (1D) vector or a two-dimensional (2D) vector, and wherein reconstructed pixels in the luma block are not used for generating the input block; and
generating a chroma prediction block of the current block by inputting the input block into the neural network-based estimating model.
2 . The method of claim 1 , wherein obtaining the reference pixels comprises:
obtaining all or some of left neighboring pixels and top neighboring pixels of the chroma block as the chroma reference pixels.
3 . The method of claim 1 , wherein obtaining the reference pixels comprises:
obtaining all or some of left neighboring pixels and top neighboring pixels of the luma block as the luma reference pixels.
4 . The method of claim 3 , wherein obtaining the reference pixels comprises:
determining locations and values of the luma reference pixels according to a color format of a current picture comprising the current block.
5 . The method of claim 1 , wherein obtaining the reference pixels comprises:
obtaining the reference pixels from multiple columns adjacent to a left side of the chroma block and a left side of the luma block, and multiple rows adjacent to a top of the chroma block and a top of the luma block.
6 . The method of claim 1 , wherein generating the input block comprises:
rearranging chroma components and luma components of the reference pixels separately and respectively based on locations of the reference pixels.
7 . The method of claim 1 , wherein the estimating model is implemented as a deep neural network comprising at least one or more neural layers and is configured to perform matrix-based operations on the input block.
8 . The method of claim 1 , wherein generating the chroma prediction block comprises:
causing the estimating model to generate the chroma prediction block to include pixels that are equal in number to pixels of the chroma block.
9 . The method of claim 1 , wherein generating the chroma prediction block comprises:
causing the estimating model to generate a reduced chroma prediction block to include pixels fewer than pixels of the chroma block.
10 . The method of claim 9 , wherein the pixels of the reduced chroma prediction block are present at locations subsampled in a row or column direction in the chroma block.
11 . The method of claim 9 , further comprising:
applying a predefined interpolation filtering to the pixels of the reduced chroma prediction block to generate an interpolated chroma prediction block having pixels that are equal in number to the pixels of the chroma block.
12 . The method of claim 1 , wherein generating the chroma prediction block comprises:
decoding, from a bitstream, a syntax element for indicating one of a plurality of pre-trained neural network-based estimating models; and
selecting, as the estimating model, one of the plurality of pre-trained neural network-based estimating models based on the syntax element.
13 . The method of claim 1 , wherein the reference pixels are selected from a group of left neighboring pixels and a group of top neighboring pixels based on a size of the current block.
14 . A method performed by a video encoding apparatus for predicting a chroma component of a current block using a luma component, the method comprising:
obtaining reference pixels that include chroma reference pixels spatially adjacent to a chroma block of the current block and include luma reference pixels adjacent to a luma block corresponding to the chroma block;
generating an input block to be inputted into a neural network-based estimating model by using the reference pixels, wherein the input block is formed as a one-dimensional (1D) vector or a two-dimensional (2D) vector, and wherein reconstructed pixels in the luma block are not used for generating the input block; and
generating a chroma prediction block of the current block by inputting the input block into the neural network-based estimating model.
15 . A method for providing a video decoding apparatus with video data, the method comprising:
encoding the video data into a bitstream; and
transmitting the bitstream to the video decoding apparatus,
wherein encoding the video data comprises:
obtaining reference pixels that include chroma reference pixels spatially adjacent to a chroma block of a current block and include luma reference pixels adjacent to a luma block corresponding to the chroma block,
generating an input block to be inputted into a neural network-based estimating model by using the reference pixels, wherein the input block is formed as a one-dimensional (1D) vector or a two-dimensional (2D) vector, and wherein reconstructed pixels in the luma block are not used for generating the input block, and
generating a chroma prediction block of the current block by inputting the input block into the neural network-based estimating model.