Interaction between IBC and BIO
Devices, systems and methods for applying intra-block copy (IBC) in video coding are described. In general, methods for integrating IBC with existing motion compensation algorithms for video encoding and decoding are described. In a representative aspect, a method for video encoding using IBC includes determining whether a current block of the current picture is to be encoded using a motion compensation algorithm, and encoding, based on the determining, the current block by selectively applying an intra-block copy to the current block. In a representative aspect, another method for video encoding using IBC includes determining whether a current block of the current picture is to be encoded using an intra-block copy, and encoding, based on the determining, the current block by selectively applying a motion compensation algorithm to the current block.
1. A method of processing video data, comprising:
determining, for a conversion between a current video block of a video region of a current picture of a video and a bitstream of the video, whether an affine prediction mode is applied to the current video block, wherein when an intra block copy (IBC) mode is applied to the current video block, the affine prediction mode is disabled for the current video block;
determining, whether Bi-directional Optical Flow (BDOF) technique is applied to the current video block at least based on τ 0 and τ 1 , wherein τ 0 is a difference between a picture order count of the current picture (current) and a picture order count (POC) of a first reference picture R 0 (τ 0 =POC(current)−POC(R 0 )), τ 1 is a difference between a picture order count of a second reference picture R 1 and the picture order count of the current picture (τ 1 =POC(R 1 )−POC(current)), wherein when a product of τ 0 and τ 1 is less than zero, the BDOF is disabled for the current video block, and when the IBC mode is applied to the current video block, the BDOF is disabled for the current video block; and
performing the conversion according to the determinings,
wherein when the IBC mode is applied to the current video block and when a spatial neighboring video block of the current video block is coded with a spatial-temporal motion vector prediction (STMVP) mode, motion information of the current video block is unavailable for deriving a corresponding video region in a collocated picture of the spatial neighboring block.
2. The method of claim 1 , wherein the BDOF technique comprises:
dividing the current video block into at least one subblock based on a size of the current video block, wherein a width and a length of the at least one subblock are both greater than or equal to 1;
obtaining a first plurality of prediction samples associated with the first reference picture and a second plurality of prediction samples associated with the second reference picture for each of the at least one subblock;
deriving at least one gradient based on the first plurality of prediction samples and the second plurality of prediction samples;
generating at least one motion vector offset based on the at least one gradient; and
generating at least one refined prediction sample for each of the at least one subblock at least based on the at least one motion vector offset.
3. The method of claim 1 , wherein the first reference picture is different from the current picture, and the second reference picture is different from the current picture.
4. The method of claim 1 , wherein the first reference picture is behind the current picture in displaying order, and the second reference picture is ahead of the current picture in displaying order.
5. The method of claim 1 , further comprising:
determining an indicator of use of the IBC technique is excluded in the bitstream in response to the BDOF technique being applied for the current video block.
6. The method of claim 1 , wherein the conversion includes encoding the video block into the bitstream.
7. The method of claim 1 , wherein the conversion includes decoding the current video block from the bitstream.
8. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
determine, for a conversion between a current video block of a video region of a current picture of a video and a bitstream of the video, whether an affine prediction mode is applied to the current video block, wherein when an intra block copy (IBC) mode is applied to the current video block, the affine prediction mode is disabled for the current video block;
determine, whether Bi-directional Optical Flow (BDOF) technique is applied to the current video block at least based on τ 0 and τ 1 , wherein τ 0 is a difference between a picture order count of the current picture (current) and a picture order count (POC) of a first reference picture R 0 (τ 0 =POC(current)−POC(R 0 )), τ 1 is a difference between a picture order count of a second reference picture R 1 and the picture order count of the current picture (τ 1 =POC(R 1 )−POC(current)), wherein when a product of τ 0 and τ 1 is less than zero, the BDOF is disabled for the current video block, and when the IBC mode is applied to the current video block, the BDOF is disabled for the current video block; and
perform the conversion according to the determinations,
wherein when the IBC mode is applied to the current video block and when a spatial neighboring video block of the current video block is coded with a spatial-temporal motion vector prediction (STMVP) mode, motion information of the current video block is unavailable for deriving a corresponding video region in a collocated picture of the spatial neighboring block.
9. The apparatus of claim 8 , wherein the BDOF technique comprises:
dividing the current video block into at least one subblock based on a size of the current video block, wherein a width and a length of the at least one subblock are both greater than or equal to 1;
obtaining a first plurality of prediction samples associated with the first reference picture and a second plurality of prediction samples associated with the second reference picture for each of the at least one subblock;
deriving at least one gradient based on the first plurality of prediction samples and the second plurality of prediction samples;
generating at least one motion vector offset based on the at least one gradient; and
generating at least one refined prediction sample for each of the at least one subblock at least based on the at least one motion vector offset.
10. The apparatus of claim 8 , wherein the first reference picture is different from the current picture, and the second reference picture is different from the current picture.
11. The apparatus of claim 8 , wherein the first reference picture is behind the current picture in displaying order, and the second reference picture is ahead of the current picture in displaying order.
12. The apparatus of claim 8 , wherein the instructions upon execution by the processor, cause the processor to:
determine an indicator of use of the IBC technique is excluded in the bitstream in response to the BDOF technique being applied for the current video block.
13. The apparatus of claim 8 , wherein the conversion includes encoding the current video block into the bitstream.
14. The apparatus of claim 8 , wherein the conversion includes decoding the current video block from the bitstream.
15. A non-transitory computer-readable storage medium storing instructions that cause a processor to:
determine, for a conversion between a current video block of a video region of a current picture of a video and a bitstream of the video, whether an affine prediction mode is applied to the current video block, wherein when an intra block copy (IBC) mode is applied to the current video block, the affine prediction mode is disabled for the current video block;
determine, whether Bi-directional Optical Flow (BDOF) technique is applied to the current video block at least based on τ 0 and τ 1 , wherein τ 0 is a difference between a picture order count of the current picture (current) and a picture order count (POC) of a first reference picture R 0 (τ 0 =POC(current)−POC(R 0 )), τ 1 is a difference between a picture order count of a second reference picture R 1 and the picture order count of the current picture (τ 1 =POC(R 1 )−POC(current)), wherein when a product of τ 0 and τ 1 is less than zero, the BDOF is disabled for the current video block, and when the IBC mode is applied to the current video block, the BDOF is disabled for the current video block; and
perform the conversion according to the determinations,
wherein when the IBC mode is applied to the current video block and when a spatial neighboring video block of the current video block is coded with a spatial-temporal motion vector prediction (STMVP) mode, motion information of the current video block is unavailable for deriving a corresponding video region in a collocated picture of the spatial neighboring block.
16. The non-transitory computer-readable storage medium of claim 15 , wherein the BDOF technique comprises:
dividing the current video block into at least one subblock based on a size of the current video block, wherein a width and a length of the at least one subblock are both greater than or equal to 1;
obtaining a first plurality of prediction samples associated with the first reference picture and a second plurality of prediction samples associated with the second reference picture for each of the at least one subblock;
deriving at least one gradient based on the first plurality of prediction samples and the second plurality of prediction samples;
generating at least one motion vector offset based on the at least one gradient; and
generating at least one refined prediction sample for each of the at least one subblock at least based on the at least one motion vector offset.
17. The non-transitory computer-readable storage medium of claim 15 , wherein the first reference picture is different from the current picture, and the second reference picture is different from the current picture.
18. The non-transitory computer-readable storage medium of claim 15 , wherein the first reference picture is behind the current picture in displaying order, and the second reference picture is ahead of the current picture in displaying order.
19. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions further cause the processor to:
determine an indicator of use of the IBC technique is excluded in the bitstream in response to the BDOF technique being applied for the current video block.
20. A non-transitory computer-readable recording medium storing a bitstream which is generated by a method performed by a video processing apparatus, wherein the method comprises:
determining, for a current video block of a video region of a current picture of a video, whether an affine prediction mode is applied to the current video block, wherein when an intra block copy (IBC) mode is applied to the current video block, the affine prediction mode is disabled for the current video block;
determining, whether Bi-directional Optical Flow (BDOF) technique is applied to the current video block at least based on τ 0 and τ 1 , wherein τ 0 is a difference between a picture order count of the current picture (current) and a picture order count (POC) of a first reference picture R 0 (τ 0 =POC(current)−POC(R 0 )), τ 1 is a difference between a picture order count of a second reference picture R 1 and the picture order count of the current picture (τ 1 =POC(R 1 )−POC(current)), wherein when a product of τ 0 and τ 1 is less than zero, the BDOF is disabled for the current video block, and when the IBC mode is applied to the current video block, the BDOF is disabled for the current video block; and
generating the bitstream from the current video block according to the determinings,
wherein when the IBC mode is applied to the current video block and when a spatial neighboring video block of the current video block is coded with a spatial-temporal motion vector prediction (STMVP) mode, motion information of the current video block is unavailable for deriving a corresponding video region in a collocated picture of the spatial neighboring block.