Subblock-based motion vector predictor with MV offset derived by template matching

1. A method of decoding performed in a decoder, the method comprising:

receiving a coded video bitstream comprising a current block in a current picture, wherein the current block includes a plurality of subblocks and is to be predicted by a subblock-based template matching motion vector prediction (SbTMVP) mode;

determining a respective collocated reference subblock for each subblock based on a combination of a displacement vector (DV) and a motion vector offset (MVO) that are associated with the respective subblock;

determining a motion vector (MV) field in the respective collocated reference subblock of each subblock in the current block;

deriving a respective reference template for each subblock based on the determined MV field of the collocated reference subblock; and

reconstructing the plurality of subblocks of the current block by predicting each subblock using the respective reference template in the SbTMVP mode.

2. The method of claim 1 , wherein the determining the respective collocated reference subblock further comprises:

determining a search area positioned in one of the current picture and a reference picture of the current picture;

determining one or more reference blocks of the current block based on template matching of a template of the current block to a template of each of the one or more reference blocks in the search area, the template of the current block including samples adjacent to the current block, the template of each of the one or more reference blocks including samples adjacent to the respective reference block of the one or more reference blocks; and

determining the respective collocated reference subblock for each subblock as a subblock that is collocated with the respective subblock in one of the one or more reference blocks.

3. The method of claim 2 , wherein the template matching of the template of the current block to the template of each of the one or more reference blocks is determined based on one of a sum of absolute difference (SAD), a sum of absolute transformed difference (SATD), a sum of squares error (SSE), a sub-sampled SAD, and a mean-removed SAD.

4. The method of claim 3 , wherein the determining the one or more reference blocks further comprises:

determining a plurality of candidate reference blocks in the search area;

determining a plurality of cost values based on the template matching of the template of the current block to templates of the plurality of candidate reference blocks; and

determining the one or more reference blocks as the one or more candidate reference blocks of the plurality of candidate reference blocks that correspond to one or more lowest cost values of the plurality of cost values.

5. The method of claim 2 , wherein the search area includes one of (i) a region centered at a position that is collocated with the current block in the reference picture and (ii) a region centered on the current block in the current picture.

6. The method of claim 2 , wherein the determining the search area further comprises:

determining the search area based on a displacement vector (DV), the DV being derived from one of (i) a motion vector of a spatially neighboring block of the current block and (ii) a motion vector of a merge candidate list of the current block.

7. The method of claim 6 , wherein the determining the search area further comprises:

determining the search area as a region centered at a sample indicated by the DV, the region being one of square-shaped, rectangular-shaped, and diamond-shaped.

8. The method of claim 6 , wherein the determining the search area further comprises:

determining the search area as a group of samples centered at a sample indicated by the DV, the group of samples being located at at least one of 0 degrees, 45 degrees, 90 degrees, or 135 degrees with respect to the sample indicated by the DV.

9. The method of claim 2 , wherein the determining the one or more reference blocks further comprises:

determining a first reference block of the one or more reference blocks that is indicated by a first displacement vector (DV) from the template of the current block to a template of the first reference block, the first DV being one of (i) derived based on the template matching such that the first DV corresponds to a cost value associated with a difference between the template of the first reference block and the template of the current block and (ii) signaled.

10. The method of claim 2 , wherein:

the determining the search area further comprises:

determining the search area based on a first displacement vector (DV) that is derived before the template matching; and

the determining the one or more reference blocks further comprises:

determining a first reference block of the one or more reference blocks that is indicated by a second DV from the template of the current block to a template of the first reference block, the second DV being derived based on the template matching such that the second DV corresponds to a cost value associated with a difference between the template of the first reference block and the template of the current block.

11. The method of claim 2 , wherein the reconstructing the plurality of subblocks of the current block further comprises:

determining one or more motion vectors (MVs) of a first subblock of the plurality of subblocks in the current block based on one or more MVs of subblocks that are collocated with the first subblock in the one or more reference blocks;

determining one or more prediction subblocks of the first subblock of the plurality of subblocks based on the one or more MVs of the first subblock; and

determining prediction samples of the first subblock based on one, or a weighted combination, of the one or more prediction subblocks.

12. The method of claim 2 , further comprising:

determining a plurality of candidate reference blocks of the current block based on a plurality of displacement vectors (DVs), each of the plurality of candidate reference blocks being indicated by a respective DV of the plurality of DVs; and

determining the one or more reference blocks of the current block from the plurality of candidate reference blocks based on one or more cost values of the template matching.

13. A method of processing visual media data, the method comprising:

processing a bitstream that includes the visual media data according to a format rule, wherein the bitstream includes a current block in a current picture, wherein the current block includes a plurality of subblocks and is to be predicted by a subblock-based template matching motion vector prediction (SbTMVP) mode; and

the format rule specifies that:

a respective collocated reference subblock for each subblock is determined based on a combination of a displacement vector (DV) and a motion vector offset (MVO) that are associated with the respective subblock;

a motion vector (MV) field in the respective collocated reference subblock of each subblock in the current block is determined;

a respective reference template for each subblock is processed based on the determined MV field of the collocated reference subblock; and

the plurality of subblocks of the current block is reconstructed by predicting each subblock using the respective reference template in the SbTMVP mode.

14. The method of claim 13 , wherein the format rule further specifies:

a search area positioned in one of the current picture and a reference picture of the current picture is determined;

one or more reference blocks of the current block is determined based on template matching of a template of the current block to a template of each of the one or more reference blocks in the search area, the template of the current block including samples adjacent to the current block, the template of each of the one or more reference blocks including samples adjacent to the respective reference block of the one or more reference blocks; and

the respective collocated reference subblock for each subblock is determined as a subblock that is collocated with the respective subblock in one of the one or more reference blocks.

15. The method of claim 14 , wherein the template matching of the template of the current block to the template of each of the one or more reference blocks is determined based on one of a sum of absolute difference (SAD), a sum of absolute transformed difference (SATD), a sum of squares error (SSE), a sub-sampled SAD, and a mean-removed SAD.

16. The method of claim 15 , wherein the format rule further specifies:

a plurality of candidate reference blocks in the search area is determined;

a plurality of cost values is determined based on the template matching of the template of the current block to templates of the plurality of candidate reference blocks; and

the one or more reference blocks is determined as the one or more candidate reference blocks of the plurality of candidate reference blocks that correspond to one or more lowest cost values of the plurality of cost values.

17. A method for video encoding in a video encoder, the method comprising:

receiving a video bitstream comprising a current block in a current picture, wherein the current block includes a plurality of subblocks and is to be predicted by a subblock-based template matching motion vector prediction (SbTMVP) mode;

processing a respective collocated reference subblock for each subblock based on a combination of a displacement vector (DV) and a motion vector offset (MVO) that are associated with the respective subblock;

processing a motion vector (MV) field in the respective collocated reference subblock of each subblock in the current block;

deriving a respective reference template for each subblock based on the processed MV field of the collocated reference subblock; and

encoding the plurality of subblocks of the current block by predicting each subblock using the respective reference template in the SbTMVP mode.

18. The method of claim 17 , wherein the method further comprises:

determining a search area positioned in one of the current picture and a reference picture of the current picture;

determining one or more reference blocks of the current block based on template matching of a template of the current block to a template of each of the one or more reference blocks in the search area, the template of the current block including samples adjacent to the current block, the template of each of the one or more reference blocks including samples adjacent to the respective reference block of the one or more reference blocks; and

determining the respective collocated reference subblock for each subblock as a subblock that is collocated with the respective subblock in one of the one or more reference blocks.

19. The method of claim 18 , wherein the template matching of the template of the current block to the template of each of the one or more reference blocks is determined based on one of a sum of absolute difference (SAD), a sum of absolute transformed difference (SATD), a sum of squares error (SSE), a sub-sampled SAD, and a mean-removed SAD.

20. The method of claim 19 , wherein the method further comprises:

determining a plurality of candidate reference blocks in the search area;

determining a plurality of cost values based on the template matching of the template of the current block to templates of the plurality of candidate reference blocks; and

determining the one or more reference blocks as the one or more candidate reference blocks of the plurality of candidate reference blocks that correspond to one or more lowest cost values of the plurality of cost values.