Affine MMVD refinement methods

1 . A method of video processing in a decoder, comprising:

extracting, from a bitstream, affine merge with motion vector difference (MMVD) candidate information of an affine MMVD candidate for a current block in a current picture, the current block being coded with an affine MMVD mode, the affine MMVD candidate including control point motion vectors (CPMVs);

determining, using the affine MMVD mode, at least a first control point motion vector (CPMV) in the CPMVs of the affine MMVD candidate based on a base predictor and a motion vector difference that is one of pre-defined motion offsets applied to the base predictor, the base predictor and the motion vector difference being indicated by the affine MMVD candidate information;

deriving, by the decoder, a first MV refinement offset associated with the first CPMV of the affine MMVD candidate according to one of pre-defined refinement positions with regard to the first CPMV;

deriving a first refined motion vector (MV) value associated with the affine MMVD candidate according to the first CPMV and the first MV refinement offset, the first refined MV value including the first MV refinement offset that is derived according to the one of the pre-defined refinement positions in addition to the base predictor and the motion vector difference that is the one of the pre-defined motion offsets applied to the base predictor; and

reconstructing a sample of the current block according to at least a first reference sample in a first reference picture, the first reference sample being indicated by the first refined MV value.

2 . The method of claim 1 , wherein the first MV refinement offset is a fraction of the motion vector difference that is applied to the base predictor to generate the affine MMVD candidate.

3 . The method of claim 2 , wherein a refinement step of the first MV refinement offset is ¼ of an affine MMVD step of the motion vector difference.

4 . The method of claim 1 , wherein the affine MMVD candidate is a uni-prediction candidate.

5 . The method of claim 1 , wherein the affine MMVD candidate is a bi-prediction candidate, and the method further comprises:

determining that a second reference picture and the first reference picture are on a same temporal side of the current picture;

deriving at least a second refined MV value associated with the affine MMVD candidate for the second reference picture, the second refined MV value being generated by applying a second MV refinement offset to a second CPMV of the affine MMVD candidate, the second MV refinement offset being equal to the first MV refinement offset, the first CPMV and the second CPMV being associated with a corner of the current block; and

reconstructing the sample according to the first reference sample in the first reference picture and a second reference sample in the second reference picture, the second reference sample being indicated by the second refined MV value.

6 . The method of claim 1 , wherein the affine MMVD candidate is a bi-prediction candidate, and the method further comprises:

determining that a second reference picture is on a different temporal side of the current picture from the first reference picture;

deriving at least a second refined MV value associated with the affine MMVD candidate for the second reference picture, the second refined MV value being generated by applying a second MV refinement offset to a second CPMV of the affine MMVD candidate, the second MV refinement offset being a mirrored offset of the first MV refinement offset, the first CPMV and the second CPMV being associated with a corner of the current block; and

reconstructing the sample according to the first reference sample in the first reference picture and a second reference sample in the second reference picture, the second reference sample being indicated by the second refined MV value.

7 . The method of claim 1 , wherein the affine MMVD candidate is a bi-prediction candidate, and the method further comprises:

determining a scaling factor based on a first temporal distance from the current picture to the first reference picture, and a second temporal distance from the current picture to a second reference picture;

deriving at least a second refined MV value associated with the affine MMVD candidate for the second reference picture, the second refined MV value being generated by applying a second MV refinement offset to a second CPMV of the affine MMVD candidate, the second MV refinement offset being a scaled offset from the first MV refinement offset according to the scaling factor, the first CPMV and the second CPMV being associated with a corner of the current block; and

reconstructing the sample according to the first reference sample in the first reference picture and a second reference sample in the second reference picture, the second reference sample being indicated by the second refined MV value.

8 . The method of claim 1 , wherein the deriving the first refined MV value further comprises:

determining the affine MMVD candidate from the affine MMVD candidate information; and

applying the derived first MV refinement offset to two or more CPMVs of the affine MMVD candidate, the two or more CPMVs being associated with different corners of the current block and including the first CPMV.

9 . The method of claim 1 , wherein the deriving the first refined MV value further comprises:

determining the affine MMVD candidate from the affine MMVD candidate information; and

applying the derived first MV refinement offset to the first CPMV that is associated with a top-left corner of the current block.

10 . The method of claim 1 , wherein the deriving the first refined MV value further comprises:

determining a second MV refinement offset corresponding to the pre-defined refinement positions; and

applying the first MV refinement offset and the second MV refinement offset respectively to the first CPMV and a second CPMV of the affine MMVD candidate, the first CPMV and the second CPMV being associated with different corners of the current block.

11 . The method of claim 1 , wherein the deriving the first refined MV value further comprises:

determining at least a second MV refinement offset according to an affine model, the affine model being a function of the one of the pre-defined refinement positions; and

applying the first MV refinement offset and the second MV refinement offset respectively to the first CPMV and a second CPMV of the affine MMVD candidate, the first CPMV and the second CPMV being associated with different corners of the current block.

12 . The method of claim 11 , wherein the one of the pre-defined refinement positions defines at least one of a base offset in the affine model, and a base ratio in the affine model.

13 . The method of claim 1 , wherein the deriving the first refined MV value further comprises:

determining the affine MMVD candidate from the affine MMVD candidate information;

applying potential MV refinements to the affine MMVD candidate to generate potential refined candidates;

calculating template matching costs respectively for the potential refined candidates; and

selecting a specific refined candidate with a best template matching cost from the potential refined candidates, the first refined MV value being one of CPMVs of the specific refined candidate.

14 . The method of claim 13 , wherein:

the affine MMVD candidate information comprises:

a first index indicative of the base predictor from a subblock based merge candidate list; and

a second index indicative of the affine MMVD candidate from a reordered list of a plurality of affine MMVD candidates, and

the determining the affine MMVD candidate from the affine MMVD candidate information comprises:

applying potential motion vector differences to the base predictor to generate the plurality of affine MMVD candidates;

calculating respective template matching costs for the plurality of affine MMVD candidates;

reordering the plurality of affine MMVD candidates into the reordered list according to the template matching costs; and

selecting the affine MMVD candidate from the reordered list according to the second index.

15 . The method of claim 1 , wherein:

the affine MMVD candidate information comprises:

a first index indicative of the base predictor from a subblock based merge candidate list; and

a second index indicative of a refined candidate from a reordered list of refined candidates, and

the deriving the first refined MV value further comprises:

applying potential motion vector differences to the base predictor to generate potential affine MMVD candidates;

applying potential MV refinement offsets respectively to each of the potential affine MMVD candidates to generate potential refined candidates for the respective one of the potential affine MMVD candidates;

determining the refined candidates respectively for the potential affine MMVD candidates according to template matching costs, a first refined candidate for a first potential affine MMVD candidate being selected from first potential refined candidates for the first potential affine MMVD candidate when the first refined candidate has a best template matching cost among the first potential refined candidates;

reordering the refined candidates to form the reordered list according to template matching costs of the refined candidates; and

selecting a specific refined candidate from the reordered list according to the second index.

16 . The method of claim 1 , further comprising:

decoding a syntax from the bitstream, the syntax indicative of whether to apply the first MV refinement offset to CPMVs of the affine MMVD candidate, the syntax being one of a sequence level syntax, a picture level syntax, a slice level syntax, a tile level syntax, a tile group level syntax, and a block level syntax.

17 . A method of video processing in an encoder, comprising:

determining to use an affine merge with motion vector difference (MMVD) candidate refinement to code a current block in a current picture, the affine MMVD candidate refinement including an affine MMVD mode;

determining, using the affine MMVD mode, a first control point motion vector (CPMV) in CPMVs of an affine MMVD candidate based on a base predictor and a motion vector difference of the affine MMVD candidate, the affine MMVD candidate including the CPMVs, the motion vector difference being one of pre-defined motion offsets applied to the base predictor;

deriving a first MV refinement offset associated with the first CPMV of the affine MMVD candidate according to one of pre-defined refinement positions with regard to the first CPMV;

deriving at least a first refined motion vector (MV) value according to the first CPMV and the first MV refinement offset, the first refined MV value including the first MV refinement offset that is derived according to the one of the pre-defined refinement positions in addition to the base predictor and the motion vector difference that is the one of the pre-defined motion offsets applied to the base predictor; and

encoding the current block based on the first refined MV value.

18 . A non-transitory computer readable storage medium storing instructions which when executed by a processor cause the processor to perform a method of encoding a bitstream comprising:

determining to use an affine merge with motion vector difference (MMVD) candidate refinement to code a current block in a current picture, the affine MMVD candidate refinement including an affine MMVD mode;

determining, using the affine MMVD mode, at least a first control point motion vector (CPMV) in CPMVs of an affine MMVD candidate based on a base predictor and a motion vector difference of the affine MMVD candidate, the affine MMVD candidate including the CPMVs, the motion vector difference being a motion offset applied to the base predictor;

deriving a first MV refinement offset associated with the first CPMV of the affine MMVD candidate according to one of pre-defined refinement positions with regard to the first CPMV;

deriving a first refined motion vector (MV) value according to the first CPMV and the first MV refinement offset, the first refined MV value including the first MV refinement offset that is derived according to the one of the pre-defined refinement positions in addition to the base predictor and the motion vector difference that is one of the pre-defined motion offsets applied to the base predictor;

encoding, in the bitstream, the current block based on the first refined MV value; and

transmitting the bitstream.