Motion compensation considering out-of-boundary conditions in video coding

1 . A method for video decoding, comprising:

deriving, by a decoder, a reference picture for a current coding block;

deriving, by the decoder, a predictor sample based on a motion vector associated with the reference picture using a motion compensation process from the reference picture;

determining, by the decoder, whether the predictor sample is located outside the reference picture by a certain margin; and

in response to determining that the predictor sample is located outside the reference picture by a certain margin, determining, by the decoder, the predictor sample is out-of-boundary (OOB),

wherein prediction refinement with optical flow (PROF) is applied in the current coding block,

wherein the predictor sample comprises a predictor sample used to derive a horizontal or vertical gradient value, and

wherein the method further comprises:

in response to determining that the predictor sample is OOB, setting an offset of the PROF associated with the predictor sample as zero; or

in response to determining that the predictor sample is OOB, setting the horizontal or vertical gradient value associated with the predictor sample as zero.

2 . The method for video decoding of claim 1 , wherein determining that the predictor sample is OOB comprises:

in response to determining that the predictor sample is a padded sample located outside the reference picture by a given sample distance, determining that the predictor sample is OOB, wherein the given sample distance is a fractional sample distance.

3 . The method for video decoding of claim 2 , wherein determining the predictor sample is OOB comprises:

determining the predictor sample is OOB, in response to determining that a sum of a horizontal position index of the predictor sample and a horizonal component of a prediction motion vector of the current coding block is not less than a sum of a position index of a right boundary of the reference picture and a first offset, M; or

determining the predictor sample is OOB, in response to determining that a sum of the horizontal position index of the predictor sample and the horizonal component of the prediction motion vector of the current coding block is not larger than a difference of a position index of a left boundary of the reference picture and a second offset, N; or

determining the predictor sample is OOB, in response to determining that a sum of a vertical position index of the predictor sample and a vertical component of the prediction motion vector of the current coding block is not less than a sum of a position index of a bottom boundary of the reference picture and a third offset, O; or

determining the predictor sample is OOB, in response to determining that a sum of the vertical position index of the predictor sample and the vertical component of the prediction motion vector of the current coding block is not larger than a difference of a position index of a top boundary of the reference picture and a fourth offset, P,

wherein the first offset, M, the second offset, N, the third offset, O, and the fourth offset P are integers.

4 . The method for video decoding of claim 1 , further comprising:

deriving a block of predictor samples for the current coding block; and

in response to determining that one predictor sample among the block of predictor samples is OOB, determining that all predictor samples among the block of predictor samples are OOB.

5 . The method for video decoding of claim 1 , wherein decoder-side motion vector refinement (DMVR) is applied in the current coding block;

wherein the predictor sample comprises a predictor sample of a candidate block in a first reference picture list or a second reference picture list;

further comprising:

in response to determining that the predictor sample of the candidate block is OOB, determining that the predictor sample of the candidate block and an associated predictor sample of a corresponding candidate block in the second reference picture list or the first reference picture list are not used to calculate a matching cost, wherein the matching cost comprises a sum of absolute sample differences or a sum of squared sample differences.

6 . The method for video decoding of claim 1 , wherein template matching (TM) is applied in the current coding block;

wherein the predictor sample comprises a predictor sample of a reference block in a reference picture for a current block template;

further comprising:

in response to determining that the predictor sample of the reference block in the reference picture for the current block template is OOB, determining that the predictor sample of the reference block in the reference picture and an associated sample of a current block template in a current picture are not used to calculate a matching cost, wherein the matching cost comprises a sum of absolute sample differences or a sum of squared sample differences.

7 . The method for video decoding of claim 1 , wherein multi-hypothesis inter prediction (MHP) mode is applied in the current coding block;

wherein the predictor sample comprises a predictor sample of an additional motion-compensated prediction;

further comprising:

in response to determining that the predictor sample of the additional motion-compensated prediction is OOB, determining that the predictor sample of the additional motion-compensated prediction is not used to generate an overall prediction signal of the MHP.

8 . The method for video decoding of claim 1 , wherein local illumination compensation (LIC) is applied in the current coding block;

wherein the predictor sample comprises a predictor sample of a reference block for a current block template;

further comprising:

in response to determining that the predictor sample of the reference block is OOB, determining that the predictor sample of the reference block and an associated sample of a current block template are not used to derive LIC parameters.

9 . The method for video decoding of claim 1 , further comprising:

receiving, by the decoder, a signaled syntax element from a bitstream to indicate whether a motion compensation process considering OOB conditions is enabled, wherein one or multiple syntax elements are signaled into the bitstream to indicate parameters or thresholds used in the motion compensation process considering OOB conditions.

10 . The method for video decoding of claim 9 , wherein the one or multiple syntax elements are signaled in sequence level, picture level, slice level, coding tree unit (CTU) or coding tree block (CTB) level, coding unit (CU) or coding block (CB) level, or prediction unit (PU) or prediction block (PB) level.

11 . The method for video decoding of claim 9 , further comprising:

receiving, by the decoder, one signaled syntax element at sequence level to indicate whether the motion compensation process considering OOB conditions is enabled;

in response to determining that the signaled syntax element indicates the motion compensation process considering OOB conditions is disabled, obtaining no more OOB related syntax elements from the bitstream; and

in response to determining that the signaled syntax element indicates the motion compensation process considering OOB conditions is enabled, obtaining OOB related syntax elements at a lower level to indicate whether the motion compensation process considering OOB conditions is enabled at the lower level.

12 . The method for video decoding of claim 9 , further comprising:

in response to determining that a sequence-level syntax element indicating the motion compensation process considering OOB conditions is enabled, obtaining OOB related syntax elements at a slice level to indicate whether the motion compensation process considering OOB conditions is enabled at the slice level.

13 . An apparatus for video coding, comprising:

one or more processors; and

a memory configured to store instructions executable by the one or more processors, wherein the one or more processors, upon execution of the instructions, are configured to perform operations, the operations comprising:

deriving a reference picture for a current coding block;

deriving a predictor sample based on a motion vector associated with the reference picture using a motion compensation process from the reference picture;

determining whether the predictor sample is located outside the reference picture by a certain margin; and

when determining that the predictor sample is located outside the reference picture by a certain margin, determining the predictor sample is out-of-boundary (OOB),

wherein prediction refinement with optical flow (PROF) is applied in the current coding block,

wherein the predictor sample comprises a predictor sample used to derive a horizontal or vertical gradient value, and

wherein the operations further comprise:

in response to determining that the predictor sample is OOB, setting an offset of the PROF associated with the predictor sample as zero; or

in response to determining that the predictor sample is OOB, setting the horizontal or vertical gradient value associated with the predictor sample as zero.

14 . The apparatus of claim 13 , wherein determining that the predictor sample is OOB comprises:

in response to determining that the predictor sample is a padded sample located outside the reference picture by a given sample distance, determining that the predictor sample is OOB, wherein the given sample distance is a fractional sample distance.

15 . The apparatus of claim 14 , wherein determining the predictor sample is OOB comprises:

determining the predictor sample is OOB, in response to determining that a sum of a horizontal position index of the predictor sample and a horizonal component of a prediction motion vector of the current coding block is not less than a sum of a position index of a right boundary of the reference picture and a first offset, M; or

determining the predictor sample is OOB, in response to determining that a sum of the horizontal position index of the predictor sample and the horizonal component of the prediction motion vector of the current coding block is not larger than a difference of a position index of a left boundary of the reference picture and a second offset, N; or

determining the predictor sample is OOB, in response to determining that a sum of a vertical position index of the predictor sample and a vertical component of the prediction motion vector of the current coding block is not less than a sum of a position index of a bottom boundary of the reference picture and a third offset, O; or

determining the predictor sample is OOB, in response to determining that a sum of the vertical position index of the predictor sample and the vertical component of the prediction motion vector of the current coding block is not larger than a difference of a position index of a top boundary of the reference picture and a fourth offset, P,

wherein the first offset, M, the second offset, N, the third offset, O, and the fourth offset P are integers.

16 . The apparatus of claim 13 , the operations further comprising:

deriving a block of predictor samples for the current coding block; and

in response to determining that one predictor sample among the block of predictor samples is OOB, determining that all predictor samples among the block of predictor samples are OOB.

17 . The apparatus of claim 13 , wherein decoder-side motion vector refinement (DMVR) is applied in the current coding block;

wherein the predictor sample comprises a predictor sample of a candidate block in a first reference picture list or a second reference picture list;

the operations further comprising:

in response to determining that the predictor sample of the candidate block is OOB, determining that the predictor sample of the candidate block and an associated predictor sample of a corresponding candidate block in the second reference picture list or the first reference picture list are not used to calculate a matching cost, wherein the matching cost comprises a sum of absolute sample differences or a sum of squared sample differences.

18 . A non-transitory computer readable storage medium storing a bitstream to be decoded and instructions which, when executed by a computing device having one or more processors, cause the one or more processors to perform operations comprising:

deriving a reference picture for a current coding block;

deriving a predictor sample based on a motion vector associated with the reference picture using a motion compensation process from the reference picture;

determining whether the predictor sample is located outside the reference picture by a certain margin; and

when determining that the predictor sample is located outside the reference picture by a certain margin, determining the predictor sample is out-of-boundary (OOB),

wherein prediction refinement with optical flow (PROF) is applied in the current coding block,

wherein the predictor sample comprises a predictor sample used to derive a horizontal or vertical gradient value, and

wherein the operations further comprise:

in response to determining that the predictor sample is OOB, setting an offset of the PROF associated with the predictor sample as zero; or

in response to determining that the predictor sample is OOB, setting the horizontal or vertical gradient value associated with the predictor sample as zero.