Concept of interweaved prediction
Methods, systems, and devices related to sub-block based motion prediction in video coding are described. In one representative aspect, a video processing method includes partitioning a video block into a first set of sub-blocks according to a first pattern, partitioning the video block into a second set of sub-blocks according to a second pattern, in which at least one sub-block in the second set has a different size than a sub-block in the first set, and determining a prediction block corresponding to a combination of a first intermediate prediction block that is predictively generated from the first set of sub-blocks and a second intermediate prediction block that is predictively generated from the second set of sub-blocks.
1. A method of video processing, comprising:
partitioning, for a conversion between a video block of a video and a bitstream of the video, the video block into a first set of sub-blocks according to a first pattern;
partitioning the video block into a second set of sub-blocks according to a second pattern, wherein at least one sub-block in the second set has a different dimension than a sub-block in the first set, and wherein the first pattern and the second pattern are determined based on a dimension of the video block;
determining a prediction block for the video block, wherein the prediction block is a combination of a first intermediate prediction block generated from the first set of sub-blocks and a second intermediate prediction block generated from the second set of sub-blocks; and
performing the conversion based on the determining.
2. The method of claim 1 , wherein the first intermediate prediction block or the second intermediate prediction block is generated using at least one of (1) an affine prediction method, (2) an alternative temporal motion vector prediction method, (3) a spatial-temporal motion vector prediction method, (4) a bi-directional optical flow method, or (5) a frame-rate up conversion method.
3. The method of claim 1 , wherein the sub-blocks in the first or the second set have a rectangular shape.
4. The method of claim 1 , wherein the sub-blocks in the first or the second set have non-uniform shapes.
5. The method of claim 1 , wherein partitioning the video block into the first set of sub-blocks is performed for a motion compensation of the video block based on a reference picture in a first reference picture list, and partitioning the video block into the second set of sub-blocks is performed for a motion compensation of the video block based on a reference picture in a second reference picture list that is different from the first reference picture list.
6. The method of claim 1 , wherein partitioning the video block into the first set of sub-blocks is performed for a motion compensation of the video block based on a reference picture in a first reference picture list, and partitioning the video block into the second set of sub-blocks is performed for a motion compensation of the video block from a reference picture in a second reference picture list that is same as the first reference picture list.
7. The method of claim 6 , wherein the motion compensation of the video block from a reference picture in the second reference picture list is performed by:
partitioning the video block into a third set of sub-blocks according to a third pattern;
generating a third intermediate prediction block based on the third set of sub-blocks;
partitioning the video block into a fourth set of sub-blocks according to a fourth pattern, wherein at least one sub-block in the fourth set has a different dimension than a sub-block in the third set;
generating a fourth intermediate prediction block based on the fourth set of sub-blocks;
determining a second prediction block based on the third intermediate prediction block and the fourth intermediate prediction block; and
determining a third prediction block based on the prediction block and the second prediction block.
8. The method of claim 1 , wherein the prediction block is determined as a weighted combination of the first intermediate prediction block weighted using a first set of weights and the second intermediate prediction block weighted using a second set of weights.
9. The method of claim 8 , wherein the first set of weights or the second set of weights includes fixed-weight values.
10. The method of claim 9 , wherein at least one value in the first set of weights is different than another value in the first set of weights.
11. The method of claim 10 , wherein at least one value in the second set of weights is different than another value in the second set of weights.
12. The method of claim 1 , further comprising performing a prediction for each sub-block in the first set of sub-blocks to determine the first intermediate prediction block.
13. The method of claim 1 , further comprising performing a prediction for each sub-block in the second set of sub-blocks to determine the second intermediate prediction block.
14. The method of claim 1 , wherein the conversion includes encoding the video block into the bitstream.
15. The method of claim 1 , wherein the conversion includes decoding the video block from the bitstream.
16. A video processing apparatus comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
partition, for a conversion between a video block of a video and a bitstream of the video, the video block into a first set of sub-blocks according to a first pattern;
partition the video block into a second set of sub-blocks according to a second pattern, wherein at least one sub-block in the second set has a different dimension than a sub-block in the first set, and wherein the first pattern and the second pattern are determined based on a dimension of the video block;
determine a prediction block for the video block, wherein the prediction block is a combination of a first intermediate prediction block generated from the first set of sub-blocks and a second intermediate prediction block generated from the second set of sub-blocks; and
perform the conversion based on the determining.
17. The video processing apparatus of claim 16 , wherein the first intermediate prediction block or the second intermediate prediction block is generated using at least one of (1) an affine prediction method, (2) an alternative temporal motion vector prediction method, (3) a spatial-temporal motion vector prediction method, (4) a bi-directional optical flow method, or (5) a frame-rate up conversion method.
18. The video processing apparatus of claim 16 , wherein the sub-blocks in the first or the second set have a rectangular shape, or the sub-blocks in the first or the second set have non-uniform shapes.
19. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:
partitioning a video block into a first set of sub-blocks according to a first pattern;
partitioning the video block into a second set of sub-blocks according to a second pattern, wherein at least one sub-block in the second set has a different dimension than a sub-block in the first set, and wherein the first pattern and the second pattern are determined based on a dimension of the video block;
determining a prediction block for the video block, wherein the prediction block is a combination of a first intermediate prediction block generated from the first set of sub-blocks and a second intermediate prediction block generated from the second set of sub-blocks; and
generating the bitstream based on the determining.