Matrix derivation in intra coding mode
Devices, systems and methods for digital video coding, which includes matrix-based intra prediction methods for video coding, are described. In a representative aspect, a method for video processing includes performing a conversion between a current video block of a video and a bitstream representation of the current video block according to a rule, where the rule specifies a relationship between samples of the current video block and matrices or offset values applied in a matrix weighted intra prediction (MIP) mode during the conversion, and where the MIP mode includes determining a prediction block of the current video block by performing, on previously coded samples of the video, a boundary downsampling operation, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation.
1. A method of processing video data, comprising:
determining, for a first conversion between a first video block of the video and a bitstream of the video, that a first intra mode is applied on the first video block of the video;
deriving reference samples of the first video block;
performing a boundary downsampling operation on the reference samples of the first video block based on a size of the first video block, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation to generate prediction samples for the first video block; and
performing the first conversion based on the prediction samples of the first video block,
wherein elements of matrices or offset values applied in the matrix vector multiplication operation have a fixed precision,
wherein the boundary downsampling operation includes deriving, according to a rule, boundary samples by applying a right bit shift operation on a sum of at least one reference sample in response to a number of shifted bits being greater than zero,
wherein the boundary samples redBdryS[x] are calculated using one of following equations:
redBdryS[ x ]=(Σ i=0 bDwn−1 refS[ x *bDwn+ i ]+(1<<(Log 2(bDwn)−1)))>>Log 2(bDwn), if bDwn>1, or
redBdryS[ x ]=refS[ x ] if bDwn=1,
wherein bDwn is equal to a function of the size of the first video block and a boundary size,
wherein refS [x] indicates a number of reference samples x,
wherein >>indicates the right bit shift operation, and
wherein <<indicates a left bit shift operation.
2. The method of claim 1 , wherein precision of the first video block and the reference samples are 10-bit depth.
3. The method of claim 1 , wherein at least two blocks with different sizes share a matrix with a same size in the first intra mode.
4. The method of claim 1 , wherein the reference samples are derived without invoking a reference sample filtering operation.
5. The method of claim 1 , wherein the reference sample filtering operation is applied in a normal intra mode which is different from the first intra mode.
6. The method of claim 1 , wherein whether the first intra mode being applied is specified by a first syntax element presented in a sequence level and a second syntax element presented in a coding unit level.
7. The method of claim 6 , wherein at least one bin of the second syntax element is context-based coded.
8. The method of claim 7 , wherein in response to a width-height ratio of the first video block being greater than 2, a context with an index of 3 is used for a first bin of the second syntax element.
9. The method of claim 7 , wherein in response to a width-height ratio of the first video block being smaller than or equal to 2, a single context selected from contexts with indices of 0, 1 or 2 is used for a first bin of the second syntax element.
10. The method of claim 1 , wherein the conversion includes encoding the current video block into the bitstream.
11. The method of claim 1 , wherein the conversion includes decoding the current video block from the bitstream.
12. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
determine, for a first conversion between a first video block of the video and a bitstream of the video, that a first intra mode is applied on the first video block of the video;
derive reference samples of the first video block;
perform a boundary downsampling operation on the reference samples of the first video block based on a size of the first video block, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation to generate prediction samples for the first video block; and
perform the first conversion based on the prediction samples of the first video block,
wherein elements of matrices or offset values applied in the matrix vector multiplication operation have a fixed precision,
wherein the boundary downsampling operation includes deriving, according to a rule, boundary samples by applying a right bit shift operation on a sum of at least one reference sample in response to a number of shifted bits being greater than zero,
wherein the boundary samples redBdryS[x] are calculated using one of following equations:
redBdryS[ x ]=(Σ i=0 bDwn−1 refS[ x *bDwn+ i ]+(1<<(Log 2(bDwn)−1)))>>Log 2(bDwn), if bDwn>1, or
redBdryS[ x ]=refS[ x ] if bDwn=1,
wherein bDwn is equal to a function of the size of the first video block and a boundary size,
wherein refS [x] indicates a number of reference samples x,
wherein >>indicates the right bit shift operation, and
wherein <<indicates a left bit shift operation.
13. The apparatus of claim 12 , wherein at least two blocks with different sizes share a matrix with a same size in the first intra mode.
14. A non-transitory computer-readable storage medium storing instructions that cause a processor to:
determine, for a first conversion between a first video block of the video and a bitstream of the video, that a first intra mode is applied on the first video block of the video;
derive reference samples of the first video block;
perform a boundary downsampling operation on the reference samples of the first video block based on a size of the first video block, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation to generate prediction samples for the first video block; and
perform the first conversion based on the prediction samples of the first video block,
wherein elements of matrices or offset values applied in the matrix vector multiplication operation have a fixed precision,
wherein the boundary downsampling operation includes deriving, according to a rule, boundary samples by applying a right bit shift operation on a sum of at least one reference sample in response to a number of shifted bits being greater than zero,
wherein the boundary samples redBdryS[x] are calculated using one of following equations:
redBdryS[ x ]=(Σ i=0 bDwn−1 refS[ x *bDwn+ i ]+(1<<(Log 2(bDwn)−1)))>>Log 2(bDwn), if bDwn>1, or
redBdryS[ x ]=refS[ x ] if bDwn=1,
wherein bDwn is equal to a function of the size of the first video block and a boundary size,
wherein refS [x] indicates a number of reference samples x,
wherein >>indicates the right bit shift operation, and
wherein <<indicates a left bit shift operation.
15. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:
determining that a first intra mode is applied on a first video block of the video;
deriving reference samples of the first video block;
performing a boundary downsampling operation on the reference samples of the first video block based on a size of the first video block, followed by a matrix vector multiplication operation, and selectively followed by an upsampling operation to generate prediction samples for the first video block; and
generating the bitstream based on the prediction samples of the first video block;
wherein elements of matrices or offset values applied in the matrix vector multiplication operation have a fixed precision,
wherein the boundary downsampling operation includes deriving, according to a rule, boundary samples by applying a right bit shift operation on a sum of at least one reference sample in response to a number of shifted bits being greater than zero,
wherein the boundary samples redBdryS[x] are calculated using one of following equations:
redBdryS[ x ]=(Σ i=0 bDwn−1 refS[ x *bDwn+ i ]+(1<<(Log 2(bDwn)−1)))>>Log 2(bDwn), if bDwn>1, or
redBdryS[ x ]=refS[ x ] if bDwn=1,
wherein bDwn is equal to a function of the size of the first video block and a boundary size,
wherein refS [x] indicates a number of reference samples x,
wherein >>indicates the right bit shift operation, and
wherein <<indicates a left bit shift operation.
16. The apparatus of claim 12 , wherein precision of the first video block and the reference samples are 10-bit depth.
17. The apparatus of claim 12 , wherein the reference samples are derived without invoking a reference sample filtering operation.
18. The apparatus of claim 12 , wherein the reference sample filtering operation is applied in a normal intra mode which is different from the first intra mode.
19. The method of claim 1 , wherein the fixed precision is an 8-bit precision.
20. The apparatus of claim 12 , wherein the fixed precision is an 8-bit precision.