Enhanced decoder side motion vector refinement
The present disclosure provides systems and methods for processing video content. The method can include: generating, for a coding block, a motion vector (MV) in a first coding mode; and updating the MV by performing a decoder side motion vector refinement (DMVR) process on the coding block.
1. A computer-implemented method for processing video content, comprising:
generating, for a coding block, a motion vector (MV) in a first coding mode, the first mode being at least one of a symmetric motion vector difference (SMVD) mode, a merge mode with motion vector differences (MMVD), a subblock-based temporal motion vector prediction (SbTMVP) mode, an affine mode, or an adaptive motion vector range (AMVR) mode; and
updating the MV by performing a decoder side motion vector refinement (DMVR) process on the coding block, wherein updating the MV by performing the DMVR process on the coding block comprises:
generating a template based on prediction blocks within one or more reference pictures associated with the MV; and
determining, based on the template, matching costs of a plurality of search points within the one or more reference pictures.
2. The method of claim 1 , wherein
the one or more reference pictures are forward reference pictures of the coding block.
3. The method of claim 1 , wherein the first coding mode is the SMVD mode, and updating the MV comprises:
in response to a first initial MV in a first list of pictures and a second initial MV in a second list of pictures being generated in the SMVD mode and the first initial MV being signaled in a bitstream,
searching for a plurality of search points in a plurality of neighboring coding blocks of the second initial MV, and
updating the second initial MV with a MV candidate associated with a search point among the plurality of search points.
4. The method of claim 1 , wherein the first coding mode is the MMVD, and updating the MV comprises:
in response to the MV being generated in the MMVD,
dividing a reference picture comprising a plurality of MVD points into a plurality of regions, wherein each region contains one MVD point, and
searching for a plurality of search points within a first region among the plurality of regions, the first region corresponding to a MVD point that is signaled in a bitstream.
5. The method of claim 1 , wherein the first coding mode is the SbTMVP mode, and updating the MV comprises:
in response to a motion shift comprised in the MV being generated in the SbTMVP mode to indicate a position of a collocated picture,
generating a plurality of templates based on a plurality of neighboring samples of the coding block and a plurality of neighboring samples of a reference block, and
searching for a plurality of MV candidates within a range of the motion shift for updating the motion shift.
6. The method of claim 1 , wherein the first coding mode is the affine mode, and updating the MV comprises:
in response to the MV being generated in the affine mode,
splitting the coding block into a plurality of subblocks, and
performing the DMVR process on the plurality of subblocks to generate a subblock MV for each of the plurality of subblocks for updating the MV based on each subblock MV.
7. The method of claim 1 , wherein the first coding mode is the AMVR mode, and updating the MV comprises:
in response to the MV being generate in the AMVR mode, searching for, based on the MV having a first resolution, a plurality of MV candidates in a second resolution, wherein the second resolution is finer than the first resolution for updating the MV.
8. An apparatus for processing video content, comprising:
a memory storing a set of instructions; and
one or more processors configured to execute the set of instructions to cause the apparatus to perform:
generating, for a coding block, a motion vector (MV) in a first coding mode, the first mode being at least one of a symmetric motion vector difference (SMVD) mode, a merge mode with motion vector differences (MMVD), a subblock-based temporal motion vector prediction (SbTMVP) mode, an affine mode, or an adaptive motion vector range (AMVR) mode; and
updating the MV by performing a decoder side motion vector refinement (DMVR) process on the coding block, wherein updating the MV by performing the DMVR process on the coding block comprises:
generating a template based on prediction blocks within one or more reference pictures associated with the MV; and
determining, based on the template, matching costs of a plurality of search points within the one or more reference pictures.
9. The apparatus of claim 8 , wherein the first coding mode is the SMVD mode, and updating the MV comprises:
in response to a first initial MV in a first list of pictures and a second initial MV in a second list of pictures being generated in the SMVD mode and the first initial MV being signaled in a bitstream,
searching for a plurality of search points in a plurality of neighboring coding blocks of the second initial MV, and
updating the second initial MV with a MV candidate associated with a search point among the plurality of search points.
10. The apparatus of claim 8 , wherein the first coding mode is the MMVD, and updating the MV comprises:
in response to the MV being generated in the MMVD,
dividing a reference picture comprising a plurality of MVD points into a plurality of regions, wherein each region contains one MVD point, and
searching for a plurality of search points within a first region among the plurality of regions, the first region corresponding to a MVD point that is signaled in a bitstream.
11. The apparatus of claim 8 , wherein the first coding mode is the SbTMVP mode, and updating the MV comprises:
in response to a motion shift comprised in the MV being generated in the SbTMVP mode to indicate a position of a collocated picture,
generating a plurality of templates based on a plurality of neighboring samples of the coding block and a plurality of neighboring samples of a reference block, and
searching for a plurality of MV candidates within a range of the motion shift for updating the motion shift.
12. The apparatus of claim 8 , wherein the first coding mode is the affine mode, and updating the MV comprises:
in response to the MV being generated in the affine mode,
splitting the coding block into a plurality of subblocks, and
performing the DMVR process on the plurality of subblocks to generate a subblock MV for each of the plurality of subblocks for updating the MV based on each subblock MV.
13. The apparatus of claim 8 , wherein the first coding mode is the AMVR mode, and updating the MV comprises:
in response to the MV being generate in the AMVR mode, searching for, based on the MV having a first resolution, a plurality of MV candidates in a second resolution, wherein the second resolution is finer than the first resolution for updating the MV.
14. A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a computer to cause the computer to perform a method for processing video content, the method comprising:
generating, for a coding block, a motion vector (MV) in a first coding mode, the first mode being at least one of a symmetric motion vector difference (SMVD) mode, a merge mode with motion vector differences (MMVD), a subblock-based temporal motion vector prediction (SbTMVP) mode, an affine mode, or an adaptive motion vector range (AMVR) mode; and
updating the MV by performing a decoder side motion vector refinement (DMVR) process on the coding block, wherein updating the MV by performing the DMVR process on the coding block comprises:
generating a template based on prediction blocks within one or more reference pictures associated with the MV; and
determining, based on the template, matching costs of a plurality of search points within the one or more reference pictures.
15. The computer readable medium of claim 14 , wherein the first coding mode is the SMVD mode, and updating the MV comprises:
in response to a first initial MV in a first list of pictures and a second initial MV in a second list of pictures being generated in the SMVD mode and the first initial MV being signaled in a bitstream,
searching for a plurality of search points in a plurality of neighboring coding blocks of the second initial MV, and
updating the second initial MV with a MV candidate associated with a search point among the plurality of search points.
16. The computer readable medium of claim 14 , wherein the first coding mode is the MMVD, and updating the MV comprises:
in response to the MV being generated in the MMVD,
dividing a reference picture comprising a plurality of MVD points into a plurality of regions, wherein each region contains one MVD point, and
searching for a plurality of search points within a first region among the plurality of regions, the first region corresponding to a MVD point that is signaled in a bitstream.
17. The computer readable medium of claim 14 , wherein the first coding mode is the SbTMVP mode, and updating the MV comprises:
in response to a motion shift comprised in the MV being generated in the SbTMVP mode to indicate a position of a collocated picture,
generating a plurality of templates based on a plurality of neighboring samples of the coding block and a plurality of neighboring samples of a reference block, and
searching for a plurality of MV candidates within a range of the motion shift for updating the motion shift.
18. The computer readable medium of claim 14 , wherein the first coding mode is the affine mode, and updating the MV comprises:
in response to the MV being generated in the affine mode,
splitting the coding block into a plurality of subblocks, and
performing the DMVR process on the plurality of subblocks to generate a subblock MV for each of the plurality of subblocks for updating the MV based on each subblock MV.
19. The computer readable medium of claim 14 , wherein the first coding mode is the AMVR mode, and updating the MV comprises:
in response to the MV being generate in the AMVR mode, searching for, based on the MV having a first resolution, a plurality of MV candidates in a second resolution, wherein the second resolution is finer than the first resolution for updating the MV.