Method for affine motion refinement
Aspects of the disclosure provide methods and apparatuses for video encoding/decoding. In some examples, an apparatus for video decoding includes processing circuitry. The processing circuitry receives a coded video bitstream including a current picture. The current picture includes a current block. The processing circuitry determines, based on decoded information from the coded video bitstream, that the current block in the current picture is coded in an affine bi-prediction mode. The current block is predicted by a plurality of control point motion vectors. The processing circuitry refines the plurality of control point motion vectors by applying bilateral matching motion refinement, and reconstructs the current block according to the refined plurality of control point motion vectors associated with the current block.
1 . A method of video decoding, the method comprising:
receiving a coded video media bitstream comprising a current picture, the current picture including a current block;
determining, based on decoded information from the coded video media bitstream, that the current block in the current picture is coded in an affine bi-prediction mode based on a first plurality of control point motion vectors of the current block associated with a first reference picture and a second plurality of control point motion vectors of the current block associated with a second reference picture;
refining one or more of the first plurality of control point motion vectors by applying bilateral matching motion refinement to the one or more of the first plurality of control point motion vectors corresponding to a non-translational part of a first affine model of the affine bi-prediction mode, the refining including:
determining a first refinement offset value for applying a first refinement on a first parameter, the first refinement updating the one or more of the first plurality of control point motion vectors to one or more first intermediate refined plurality of control point motion vectors according to a bilateral matching cost that is based on the first refinement on the first parameter; and
determining a second refinement offset value for applying a second refinement on a second parameter, the second refinement updating the one or more first intermediate refined plurality of control point motion vectors to one or more second intermediate affined plurality of control point motion vectors according to a bilateral matching cost that is based on the second refinement on the second parameter; and
reconstructing the current block according to the refined one or more of the first plurality of control point motion vectors associated with the current block.
2 . The method of claim 1 , further comprising:
determining a second plurality of motion vector refinement offset values for the second plurality of control point motion vectors associated with the second reference picture according to the first refinement offset value and the second refinement offset value, the second plurality of motion vector refinement offset values having opposite signs from the first refinement offset value and the second refinement offset value.
3 . The method of claim 1 , further comprising:
generating a second set of affine parameter refinement offset values for second affine parameters of a second affine model of the current block associated with the second reference picture according to the first refinement offset value and the second refinement offset value, the second set of affine parameter refinement offset values being of opposite signs from the first refinement offset value and the second refinement offset value.
4 . The method of claim 3 , wherein the first affine model and the second affine model are 4-parameter affine models or 6-parameter affine models.
5 . The method of claim 1 , wherein the first plurality of control point motion vectors is determined based on at least one of:
an affine merge candidate; and
an affine merge motion vector difference (MMVD) candidate.
6 . The method of claim 1 , wherein at least the first parameter and the second parameter for refinement comprise a first one of the one or more of the first plurality of control point motion vectors of a first control point and a second one of the one or more of the first plurality of control point motion vectors of a second control point.
7 . The method of claim 1 , wherein at least the first parameter and the second parameter for refinement comprise a first affine parameter and a second affine parameter of corresponding to the non-translational part of the first affine model.
8 . The method of claim 1 , wherein the refining the one or more of the first plurality of control point motion vectors comprises:
applying affine motion compensation to subblocks of the current block to determine first reference subblocks in the first reference picture and second reference subblocks in the second reference picture corresponding to the subblocks of the current block, the affine motion compensation being based on the one or more first intermediate refined plurality of control point motion vectors; and
calculating the bilateral matching cost associated with the first refinement on the first parameter based on a distortion between the first reference subblocks and the second reference subblocks.
9 . The method of claim 1 , wherein the refining the one or more of the first plurality of control point motion vectors comprises:
applying affine motion compensation to a subset of subblocks of the current block to determine a first subset of reference subblocks in the first reference picture and a second subset of reference subblocks in the second reference picture corresponding to the subset of subblocks of the current block, the affine motion compensation being based on the one or more first intermediate refined plurality of control point motion vectors; and
calculating the bilateral matching cost associated with the first refinement on the first parameter based on a distortion between the first subset of reference subblocks and the second subset of reference subblocks.
10 . The method of claim 9 , wherein the subset of subblocks in the current block comprises a subblock at a center position of the current block.
11 . The method of claim 9 , wherein the subset of subblocks in the current block comprises one or more subblocks at one or more corners of the current block.
12 . The method of claim 1 , further comprising:
using a first affine motion compensation subblock size during the bilateral matching motion refinement; and
reconstructing the current block using a second affine motion compensation subblock size, the first affine motion compensation subblock size being larger than the second affine motion compensation subblock size.
13 . The method of claim 1 , wherein
the first plurality of control point motion vectors includes the one or more of the first plurality of control point motion vectors and a third control point motion vector;
the first affine model includes a translational part corresponding to the third control point motion vector of a third control point of the current block and the non-translational part corresponding to the one or more of the first plurality of control point motion vectors of one or more first control points of the current block; and
the bilateral matching motion refinement is not applied to refine the third control point motion vector corresponding to the translational part.
14 . The method of claim 13 , wherein
the third control point of the current block is at a top-left corner of the current block.
15 . A method of video encoding, the method comprising:
refining one or more of a first plurality of control point motion vectors by applying bilateral matching motion refinement to the one or more of the first plurality of control point motion vectors corresponding to a non-translational part of a first affine model, a current block being encoded with an affine bi-prediction mode based on the first plurality of control point motion vectors of the current block associated with a first reference picture and a second plurality of control point motion vectors of the current block associated with a second reference picture; and
encoding the current block according to the refined one or more of the first plurality of control point motion vectors associated with the current block, wherein
the refining includes:
determining a first refinement offset value for applying a first refinement on a first parameter, the first refinement updating the one or more of the first plurality of control point motion vectors to one or more first intermediate refined plurality of control point motion vectors according to a bilateral matching cost that is based on the first refinement on the first parameter; and
determining a second refinement offset value for applying a second refinement on a second parameter, the second refinement updating the one or more first intermediate refined plurality of control point motion vectors to one or more second intermediate affined plurality of control point motion vectors according to a bilateral matching cost that is based on the second refinement on the second parameter.
16 . A non-transitory computer readable medium storing a video media bitstream encoded by an encoding method, the encoding method comprising:
refining one or more of a first plurality of control point motion vectors by applying bilateral matching motion refinement to the one or more of the first plurality of control point motion vectors corresponding to a non-translational part of a first affine model, a current block being encoded with an affine bi-prediction mode based on the first plurality of control point motion vectors of the current block associated with a first reference picture and a second plurality of control point motion vectors of the current block associated with a second reference picture; and
encoding the current block according to the refined one or more of the first plurality of control point motion vectors associated with the current block, wherein
the refining includes:
determining a first refinement offset value for applying a first refinement on a first parameter, the first refinement updating the one or more of the first plurality of control point motion vectors to one or more first intermediate refined plurality of control point motion vectors according to a bilateral matching cost that is based on the first refinement on the first parameter; and
determining a second refinement offset value for applying a second refinement on a second parameter, the second refinement updating the one or more first intermediate refined plurality of control point motion vectors to one or more second intermediate affined plurality of control point motion vectors according to a bilateral matching cost that is based on the second refinement on the second parameter.