Methods and systems for performing combined inter and intra prediction
A method for performing combined inter and intra prediction (CIIP) includes determining the CIIP being enabled for a target block; determining a first intra prediction mode of the target block using a template-based intra mode derivation (TIMD) method; generating an intra predictor of the target block with the first intra prediction mode; and obtaining a final predictor of the target block by weighted averaging the intra predictor of the target block and an inter predictor of the target block.
1. A method for performing combined inter and intra prediction (CIIP), comprising:
determining the CIIP being enabled for a target block;
determining a first intra prediction mode of the target block using a template-based intra mode derivation (TIMD) method;
generating an intra predictor of the target block with the first intra prediction mode; and
obtaining a final predictor of the target block by weighted averaging the intra predictor of the target block and an inter predictor of the target block;
wherein obtaining the final predictor of the target by weighted averaging the intra predictor of the target block and the inter predictor of the target block further comprises:
in response to the first intra prediction mode being an angular mode, determining an intra weight and an inter weight based on the first intra prediction mode; and
obtaining the final predictor of the target block by weighted averaging the intra predictor of the target block and the inter predictor of the target block with the intra weight and the inter weight, respectively;
wherein determining the intra weight and the inter weight based on the first intra prediction mode further comprises:
dividing the target block into a plurality of sub-blocks, wherein the target block is divided vertically when an angular mode index of the first intra prediction mode is less than a preset value, or the target block is divided horizontally when the angular mode index of the first intra prediction mode is equal to or greater than the preset value; and
determining a sub-intra weight and a sub-inter weight for each of the plurality of sub-blocks; and
wherein obtaining the final predictor of the target block by weighted averaging the intra predictor of the target block and the inter predictor of the target block further comprises:
determining a plurality of sub-final predictors associated with the plurality of sub-blocks, respectively, wherein each of the plurality of sub-final predictors is determined by weighted averaging the intra predictor and the inter predictor with the sub-intra weight and sub-inter weight for the respective sub-block;
determining a sum of the plurality of sub-final predictors; and
obtaining the final predictor by right-shifting a number of bits, wherein the number of bits is obtained by logarithm of a sum of the sub-intra weight and the sub-inter weight.
2. The method according to claim 1 , wherein determining the first intra prediction mode of the target block using the TIMD method further comprises:
calculating values of sum of absolute transformed differences (SATD) of the target block associated respectively with a plurality of intra prediction modes in a TIMD mode list; and
determining, among the plurality of intra prediction modes, an intra prediction mode with a smallest SATD value as the first intra prediction mode of the target block.
3. The method according to claim 1 , wherein determining the first intra prediction mode of the target block using the TIMD method further comprises:
calculating values of SATD of the target block associated respectively with a plurality of intra prediction modes in a TIMD mode list;
determining, among the plurality of intra prediction modes, an intra prediction mode with a smallest SATD value, and mapping the intra prediction mode with the smallest SATD value to a regular intra prediction mode; and
determining the regular intra prediction mode as the first intra prediction mode of the target block.
4. The method according to claim 1 , wherein:
the target block is divided vertically into four equally sized sub-blocks when the angular mode index of the first intra prediction mode is equal to or greater than an angular mode index of a diagonal mode from bottom left to top right and less than an angular mode index of the diagonal mode from top left to bottom right, or the target block is divided horizontally into four equally sized sub-blocks when the angular mode index of the first intra prediction mode is equal to or greater than the angular mode index of the diagonal mode from top left to bottom right and equal to or less than an angular mode index of the diagonal mode from top right to bottom left;
the four equally sized sub-blocks include a first sub-block, a second sub-block, a third sub-block and a fourth sub-block, the first, second, third, and fourth sub-blocks being arranged respectively: from left to right when the target block is divided vertically, or from top to bottom when the target block is divided horizontally; and
the sub-intra weight and sub-inter weight for the first sub-block are 6 and 2, respectively;
the sub-intra weight and sub-inter weight for the second sub-block are 5 and 3, respectively;
the sub-intra weight and sub-inter weight for the third sub-block are 3 and 5, respectively; and
the sub-intra weight and sub-inter weight for the fourth sub-block are 2 and 6, respectively.
5. The method according to claim 1 , wherein determining the first intra prediction mode of the target block using the TIMD method further comprises:
when the target block has a size less than or equal to a threshold, determining the first intra prediction mode of the target block using the TIMD method; and
when the target block has a size greater than a threshold, determining the first intra prediction mode of the target block as a Planar mode.
6. The method according to claim 5 , wherein the threshold is equal to 1024.
7. The method according to claim 1 , further comprising:
setting a planar mode as a propagated intra prediction mode; and
constructing a most probable mode (MPM) list for a neighboring block of the target block with the propagated intra prediction mode.
8. A method for performing combined inter and intra prediction (CIIP), comprising:
determining a first intra prediction mode of a target block using a template-based intra mode derivation (TIMD) method;
generating an intra predictor of the target block with the first intra prediction mode;
obtaining a final predictor of the target block by weighted averaging the intra predictor of the target block and an inter predictor of the target block; and
signaling a flag indicating the CIIP being enabled and an index indicating the TIMD method is used for determining the first intra prediction mode of the target block;
wherein obtaining the final predictor of the target by weighted averaging the intra predictor of the target block and the inter predictor of the target block further comprises:
in response to the first intra prediction mode being an angular mode, determining an intra weight and an inter weight based on the first intra prediction mode; and
obtaining the final predictor of the target block by weighted averaging the intra predictor of the target block and the inter predictor of the target block with the intra weight and the inter weight, respectively;
wherein determining the intra weight and the inter weight based on the first intra prediction mode further comprises:
dividing the target block into a plurality of sub-blocks, wherein the target block is divided vertically when an angular mode index of the first intra prediction mode is less than a preset value, or the target block is divided horizontally when the angular mode index of the first intra prediction mode is equal to or greater than the preset value; and
determining a sub-intra weight and a sub-inter weight for each of the plurality of sub-blocks; and
wherein obtaining the final predictor of the target block by weighted averaging the intra predictor of the target block and the inter predictor of the target block further comprises:
determining a plurality of sub-final predictors associated with the plurality of sub-blocks, respectively, wherein each of the plurality of sub-final predictors is determined by weighted averaging the intra predictor and the inter predictor with the sub-intra weight and sub-inter weight for the respective sub-block;
determining a sum of the plurality of sub-final predictors; and
obtaining the final predictor by right-shifting a number of bits, wherein the number of bits is obtained by logarithm of a sum of the sub-intra weight and the sub-inter weight.
9. The method according to claim 8 , wherein determining the first intra prediction mode of the target block using the TIMD method further comprises:
calculating values of sum of absolute transformed differences (SATD) of the target block associated respectively with a plurality of intra prediction modes in a TIMD mode list; and
determining, among the plurality of intra prediction modes, an intra prediction mode with a smallest SATD value as the first intra prediction mode of the target block.
10. The method according to claim 8 , wherein determining the first intra prediction mode of the target block using the TIMD method further comprises:
calculating values of SATD of the target block associated respectively with a plurality of intra prediction modes in a TIMD mode list;
determining, among the plurality of intra prediction modes, an intra prediction mode with a smallest SATD value, and mapping the intra prediction mode with the smallest SATD value to a regular intra prediction mode; and
determining the regular intra prediction mode as the first intra prediction mode of the target block.
11. The method according to claim 8 , wherein:
the target block is divided vertically into four equally sized sub-blocks when the angular mode index of the first intra prediction mode is equal to or greater than an angular mode index of a diagonal mode from bottom left to top right and less than an angular mode index of the diagonal mode from top left to bottom right, or the target block is divided horizontally into four equally sized sub-blocks when the angular mode index of the first intra prediction mode is equal to or greater than the angular mode index of the diagonal mode from top left to bottom right and equal to or less than an angular mode index of the diagonal mode from top right to bottom left;
the four equally sized sub-blocks include a first sub-block, a second sub-block, a third sub-block and a fourth sub-block, the first, second, third, and fourth sub-blocks being arranged respectively: from left to right when the target block is divided vertically, or from top to bottom when the target block is divided horizontally; and
the sub-intra weight and sub-inter weight for the first sub-block are 6 and 2, respectively;
the sub-intra weight and sub-inter weight for the second sub-block are 5 and 3, respectively;
the sub-intra weight and sub-inter weight for the third sub-block are 3 and 5, respectively; and
the sub-intra weight and sub-inter weight for the fourth sub-block are 2 and 6, respectively.
12. The method according to claim 8 , wherein determining the first intra prediction mode of the target block using the TIMD method further comprises:
when the target block has a size less than or equal to a threshold, determining the first intra prediction mode of the target block using the TIMD method; and
when the target block has a size greater than a threshold, determining the first intra prediction mode of the target block as a Planar mode.
13. The method according to claim 12 , wherein the threshold is equal to 1024.
14. The method according to claim 8 , further comprising:
setting a planar mode as a propagated intra prediction mode; and
constructing a most probable mode (MPM) list for a neighboring block of the target block with the propagated intra prediction mode.
15. A non-transitory computer readable medium storing a bitstream, wherein the bitstream comprises a flag and a first index associated with encoded video data, the flag indicating an inter and intra prediction (CIIP) is used for the encoded video data, and the first index indicating a template-based intra mode derivation (TIMD) method used for the CIIP, wherein the flag and the index cause a decoder to:
determine a first intra prediction mode of a target block using the TIMD method;
generate an intra predictor of the target block with the intra prediction mode; and
obtain a final predictor of the target block by weighted averaging the intra predictor of the target block and an inter predictor of the target block;
wherein the bitstream further comprises an angular mode index associated with the video data, and the angular mode index cause the decoder to:
determine an intra weight and an inter weight based on the first intra prediction mode; and
obtain the final predictor of the target block by weighted averaging the intra predictor of the target block and the inter predictor of the target block with the intra weight and the inter weight, respectively; wherein the angular mode index further causes the decoder to:
divide the target block into a plurality of sub-blocks, wherein the target block is divided vertically when the angular mode index of the first intra prediction mode is less than a preset value, or the target block is divided horizontally when the angular mode index of the first intra prediction mode is equal to or greater than the preset value; and
determine a sub-intra weight and a sub-inter weight for each of the plurality of sub-blocks;
determine a plurality of sub-final predictors associated with the plurality of sub-blocks, respectively, wherein each of the plurality of sub-final predictors is determined by weighted averaging the intra predictor and the inter predictor with the sub-intra weight and sub-inter weight for the respective sub-block;
determine a sum of the plurality of sub-final predictors; and
obtain the final predictor by right-shifting a number of bits, wherein the number of bits is obtained by logarithm of a sum of the sub-intra weight and the sub-inter weight.
16. The non-transitory computer readable medium according to claim 15 , wherein the angular mode index causes the decoder to:
vertically divide the target block into four equally sized sub-blocks when the angular mode index of the first intra prediction mode is equal to or greater than an angular mode index of a diagonal mode from bottom left to top right and less than an angular mode index of the diagonal mode from top left to bottom right, or horizontally divide the target block into four equally sized sub-blocks when the angular mode index of the first intra prediction mode is equal to or greater than the angular mode index of the diagonal mode from top left to bottom right and equal to or less than an angular mode index of the diagonal mode from top right to bottom left;
wherein the four equally sized sub-blocks include a first sub-block, a second sub-block, a third sub-block and a fourth sub-block, the first, second, third, and fourth sub-blocks being arranged respectively: from left to right when the target block is divided vertically, or from top to bottom when the target block is divided horizontally; and
the sub-intra weight and sub-inter weight for the first sub-block are 6 and 2, respectively;
the sub-intra weight and sub-inter weight for the second sub-block are 5 and 3, respectively;
the sub-intra weight and sub-inter weight for the third sub-block are 3 and 5, respectively; and
the sub-intra weight and sub-inter weight for the fourth sub-block are 2 and 6, respectively.