Intra prediction for square and non-square blocks in video compression
In a method for video decoding, a determination is made as to whether an angular intra prediction mode for a current block which is a non-square block is a wide angle mode outside a range of directions spanning a bottom left diagonal direction and a top right diagonal direction of the current block. When the angular intra prediction mode is the wide angle mode, whether a condition for applying an intra smoothing filter to blocks neighboring the current block is met is determined. When the condition is met, the intra smoothing filter is applied to the neighboring blocks and intra prediction is performed with a coding tool based on the filtered blocks to obtain a feature value for the current block. When the coding tool is a Non-Separable Secondary Transform, the wide angle mode is mapped to a conventional intra prediction mode within the range of directions.
1 . A method for video decoding, comprising:
for a current block which is a non-square block, determining whether an angular intra prediction mode for the current block is a wide angle mode that is in a direction outside a range of directions spanning a bottom left diagonal direction and a top right diagonal direction of the current block;
when the angular intra prediction mode is determined to be the wide angle mode, determining whether a condition for applying an intra smoothing filter to blocks neighboring the current block is met, the condition being met when a block size of the non-square block is greater than or equal to a block size threshold; and
when the condition is determined to be met,
applying the intra smoothing filter to the blocks neighboring the current block;
performing intra prediction with a coding tool based on the filtered blocks to decode the current block and to obtain a feature value for the current block;
determining whether the coding tool used in the intra prediction is a Non-Separable Secondary Transform (NSST); and
when the coding tool is determined to be the NSST, mapping the wide angle mode to a conventional intra prediction mode within the range of directions spanning the bottom left diagonal direction and the top right diagonal direction of the current block, the wide angle mode being mapped to an intra prediction mode in the range of directions that is nearest to the wide angle mode.
2 . The method according to claim 1 , wherein the block size is determined according to a height, a width, or an average of the height and the width of the current block.
3 . The method according to claim 1 , wherein
there are 65 conventional angular intra prediction modes corresponding to mode numbers 2 to 66, respectively,
when a mode number of the wide angle mode is one of 67 to 74, the wide angle mode is mapped to the conventional intra prediction mode that is a conventional angular intra prediction corresponding to the mode number 66, and
when the mode number of the wide angle mode is one of −8 to −1, the wide angle mode is mapped to the conventional intra prediction mode that is a conventional angular intra prediction corresponding to the mode number 2.
4 . A method for video encoding, comprising:
for a current block which is a non-square block, determining whether an angular intra prediction mode for the current block is a wide angle mode that is in a direction outside a range of directions spanning a bottom left diagonal direction and a top right diagonal direction of the current block;
when the angular intra prediction mode is determined to be the wide angle mode, determining whether a condition for applying an intra smoothing filter to blocks neighboring the current block is met, the condition being met when a block size of the non-square block is greater than or equal to a block size threshold; and
when the condition is determined to be met,
applying the intra smoothing filter to the blocks neighboring the current block;
performing intra prediction with a coding tool based on the filtered blocks to encode the current block and to obtain a feature value for the current block;
determining whether the coding tool used in the intra prediction is a Non-Separable Secondary Transform (NSST); and
when the coding tool is determined to be the NSST, mapping the wide angle mode to a conventional intra prediction mode within the range of directions spanning the bottom left diagonal direction and the top right diagonal direction of the current block, the wide angle mode being mapped to an intra prediction mode in the range of directions that is nearest to the wide angle mode.
5 . The method according to claim 4 , wherein the block size is determined according to a height, a width, or an average of the height and the width of the current block.
6 . The method according to claim 4 , wherein
there are 65 conventional angular intra prediction modes corresponding to mode numbers 2 to 66, respectively,
when a mode number of the wide angle mode is one of 67 to 74, the wide angle mode is mapped to the conventional intra prediction mode that is a conventional angular intra prediction corresponding to the mode number 66, and
when the mode number of the wide angle mode is one of −8 to −1, the wide angle mode is mapped to the conventional intra prediction mode that is a conventional angular intra prediction corresponding to the mode number 2.
7 . A non-transitory computer-readable storage medium storing instructions which when executed by a processor cause the processor to perform a method of encoding a bitstream comprising:
for a current block which is a non-square block, determining whether an angular intra prediction mode for the current block is a wide angle mode that is in a direction outside a range of directions spanning a bottom left diagonal direction and a top right diagonal direction of the current block;
when the angular intra prediction mode is determined to be the wide angle mode, determining whether a condition for applying an intra smoothing filter to blocks neighboring the current block is met, the condition being met when a block size of the non-square block is greater than or equal to a block size threshold; and
when the condition is determined to be met,
applying the intra smoothing filter to the blocks neighboring the current block;
performing intra prediction with a coding tool based on the filtered blocks to encode the current block and to obtain a feature value for the current block;
determining whether the coding tool used in the intra prediction is a Non-Separable Secondary Transform (NSST); and
when the coding tool is determined to be the NSST, mapping the wide angle mode to a conventional intra prediction mode within the range of directions spanning the bottom left diagonal direction and the top right diagonal direction of the current block, the wide angle mode being mapped to an intra prediction mode in the range of directions that is nearest to the wide angle mode; and
transmitting the bitstream including the encoded current block.
8 . The method according to claim 7 , wherein the block size is determined according to a height, a width, or an average of the height and the width of the current block.
9 . The method according to claim 7 , wherein
there are 65 conventional angular intra prediction modes corresponding to mode numbers 2 to 66, respectively,
when a mode number of the wide angle mode is one of 67 to 74, the wide angle mode is mapped to the conventional intra prediction mode that is a conventional angular intra prediction corresponding to the mode number 66, and
when the mode number of the wide angle mode is one of −8 to −1, the wide angle mode is mapped to the conventional intra prediction mode that is a conventional angular intra prediction corresponding to the mode number 2.