Bilateral matching and decoder side motion vector refinement with out-of-frame boundary condition
A method of video decoding includes receiving a coded video bitstream comprising coded information of a current block in a current picture. The coded information is indicative of a bi-directional prediction with a refinement. The refinement determines an offset based on a plurality of calculations, and each calculation includes reference samples of a first reference picture and a second reference picture for the current picture. The method further includes determining that a first calculation in the plurality of calculations uses a first reference sample that is an out of boundary (OOB) prediction sample. An OOB prediction sample of a reference picture is out of a frame boundary of the reference picture. The method also includes performing the refinement that determines the offset based on the plurality of calculations with at least the first reference sample being excluded, and reconstructing the current block according to the bi-directional prediction with the offset.
1 . A method of video decoding, comprising:
receiving a coded video bitstream comprising coded information of a current block in a current picture, the coded information indicative of a decoder-side motion vector refinement (DMVR) that determines a motion vector offset based on motion vector offset candidates, and each motion vector offset candidate corresponding to reference samples associated with a first reference picture for the current picture and a second reference picture for the current picture;
determining that a first reference block corresponding to a first motion vector offset candidate in the motion vector offset candidates includes a first reference sample that is an out of boundary (OOB) prediction sample of the first reference picture, the OOB prediction sample of the first reference picture being out of a frame boundary of the first reference picture, the first motion vector offset candidate corresponding to the first reference block and a second reference block associated with the second reference picture;
performing the DMVR that determines the motion vector offset based on a plurality of calculations, wherein (i) the first reference block including the OOB prediction sample and (ii) the second reference block are not used in the plurality of calculations; and
reconstructing the current block according to the DMVR with the motion vector offset.
2 . The method of claim 1 , wherein the performing the DMVR further comprises: determining bilateral matching (BM) cost values associated with the motion vector offset candidates.
3 . The method of claim 2 , wherein the determining the BM cost values comprises:
determining one of the BM cost values as a distortion measure between corresponding reference samples in a pair of reference blocks that are associated with a second motion vector offset candidate, the pair of reference blocks being located within the first reference picture and the second reference picture, respectively.
4 . The method of claim 2 , wherein the determining the BM cost values further comprises at least one of:
assigning a maximum BM cost value in the BM cost values as associated with the first motion vector offset candidate; and/or
marking the first motion vector offset candidate as unavailable.
5 . The method of claim 2 , wherein the first motion vector offset candidate corresponds to an integer position that is a neighboring position of a specific position having a lowest BM cost, and the method further comprises:
determining to use a fractional block matching to replace quadratic prediction for a fractional sample refinement at the specific position; and
performing the fractional sample refinement for the specific position by calculating BM cost values at positions corresponding to fractional sample offsets.
6 . The method of claim 1 , wherein the current block is a coding block or a subblock of the coding block.
7 . A method of video encoding, comprising:
determining that a first reference block corresponding to a first motion vector offset candidate in motion vector offset candidates includes a first reference sample that is an out of boundary (OOB) prediction sample of a first reference picture for a current picture, the OOB prediction sample of the first reference picture being out of a frame boundary of the first reference picture, the first motion vector offset candidate corresponding to the first reference block and a second reference block associated with a second reference picture for the current picture, a current block in the current picture being coded using a decoder-side motion vector refinement (DMVR) that determines a motion vector offset based on the motion vector offset candidates, and each motion vector offset candidate corresponding to reference samples associated with the first reference picture and the second reference picture for the current picture;
performing the DMVR that determines the motion vector offset based on a plurality of calculations, wherein (i) the first reference block including the OOB prediction sample and (ii) the second reference block are not used in the plurality of calculations; and
encoding, in a bitstream, (i) the current block according to the DMVR with the motion vector offset and (ii) coded information of the current block that indicates the DMVR.
8 . The method of claim 7 , wherein the performing the DMVR further comprises: determining bilateral matching (BM) cost values associated with the motion vector offset candidates.
9 . The method of claim 8 , wherein the determining the BM cost values comprises:
determining one of the BM cost values as a distortion measure between corresponding reference samples in a pair of reference blocks that are associated with a second motion vector offset candidate, the pair of reference blocks being located within the first reference picture and the second reference picture, respectively.
10 . The method of claim 8 , wherein the determining the BM cost values further comprises at least one of:
assigning a maximum BM cost value in the BM cost values as associated with the first motion vector offset candidate; and/or
marking the first motion vector offset candidate as unavailable.
11 . The method of claim 8 , wherein the first motion vector offset candidate corresponds to an integer position that is a neighboring position of a specific position having a lowest BM cost, and the method further comprises:
determining to use a fractional block matching to replace quadratic prediction for a fractional sample refinement at the specific position; and
performing the fractional sample refinement for the specific position by calculating BM cost values at positions corresponding to fractional sample offsets.
12 . A non-transitory computer-readable storage medium storing instructions which when executed by a processor cause the processor to perform a method of encoding a bitstream comprising:
determining that a first reference block corresponding to a first motion vector offset candidate in motion vector offset candidates includes a first reference sample that is an out of boundary (OOB) prediction sample of a first reference picture for a current picture, the OOB prediction sample of the first reference picture being out of a frame boundary of the first reference picture, the first motion vector offset candidate corresponding to the first reference block and a second reference block associated with a second reference picture, a current block in the current picture being coded using a decoder-side motion vector refinement (DMVR) that determines a motion vector offset based on the motion vector offset candidates, and each motion vector offset candidate corresponding to reference samples associated with the first reference picture and the second reference picture for the current picture;
performing the DMVR that determines the motion vector offset based on a plurality of calculations wherein (i) the first reference block including the OOB prediction sample and (ii) the second reference block are not used in the plurality of calculations;
encoding, in the bitstream, (i) the current block according to the DMVR with the motion vector offset and (ii) coded information of the current block that indicates the DMVR; and
transmitting the encoded bitstream.