Conditional application of inter prediction with geometric partitioning in video processing
A method for processing a video includes performing a determination, by a processor, that a first video block is partitioned to include a first prediction portion that is non-rectangular and non-square; adding a first motion vector (MV) prediction candidate associated with the first prediction portion to a motion candidate list associated with the first video block, wherein the first MV prediction candidate is derived from a sub-block MV prediction candidate; and performing further processing of the first video block using the motion candidate list.
1 . A method of video processing, comprising:
making a determination that a first conversion between a first video block of a video region of a video and a bitstream of the video uses a geometry partition mode in which the first video block is partitioned into multiple prediction partitions including at least a first prediction partition;
deriving N motion information candidates using one or more sub-block merge candidates associated with the first video block and adding the N motion information candidates to a motion candidate list for the first video block, where N is a positive integer;
deriving, from the motion candidate list, motion candidates for the multiple prediction partitions; and
performing the first conversion based on the motion candidates for the multiple prediction partitions.
2 . The method of claim 1 , wherein a sub-block temporal motion vector predictor candidate or an affine motion candidate is used for deriving the N motion information candidates, or
wherein M representative motion information candidates from one or more sub-block motion candidates of one or more sub-blocks are used for deriving the M motion information candidates, where M is a positive integer, and wherein the M representative motion information candidates are selected based positions of the one or more sub-blocks or selected from motion information associated with a center sub-block of the first prediction partition, wherein M is equal to a number of partitions in which the first video block is partitioned for the first conversion, or M=1, or
wherein a sub-block of the prediction partition is assigned a sub-block motion information thereof, and wherein the sub-block motion information is used as a candidate in a candidate list used for the first conversion of the first video block.
3 . The method of claim 1 , further comprising:
adding a first motion information candidate to a first candidate list associated with the first prediction partition and a second motion information candidate to a second candidate list associated with a second prediction partition, wherein the first motion information candidate and the second motion information candidate are inherited from a single motion candidate; and
performing the first conversion using the first motion information candidate and/or the second motion information candidate,
wherein a range of indication of splitting patterns and MV candidate indices for the first prediction partition and the second prediction partition is M*N*N, wherein M is a number of prediction portions that the first video block is partitioned into, and wherein N is a number of allowed merge candidates to be added to a motion candidate list including the first motion information candidate and the second motion information candidate, and wherein a specific weighting factor group is used for samples located in a weighted area of the first video block according to the splitting pattern, wherein the specific weighting factor group is different from another weighting factor group used for another video block in which different motion candidates are used for determining candidate motion vectors for prediction portions of the another video block, and wherein the specific weighting factor group corresponds to averaging for the samples located in the weighted area.
4 . The method of claim 3 , wherein (i) performing the first conversion includes deriving a refined final motion vector for a prediction block for the first video block, wherein the deriving the refined final motion vector includes using a decoder-side motion vector refinement (DMVR) tool to a motion information of the first prediction partition and a motion information of the second prediction partition of the first video block to derive motion information for the first video block, wherein the first prediction partition is uni-predicted from a first reference list and the second prediction partition is bi- or uni-predicted using a second reference list, then the DMVR tool or a bi-directional optical flow (BIO) tool are applied using motion information of the second prediction partitions in the second reference list, and wherein the first conversion uses a final prediction that is a weighted sum of a prediction result based on the geometry partition mode and a prediction result based on an intra-prediction mode, or (ii) the first conversion comprises deriving one or more motion vector prediction candidates in a motion candidate list from a sub-block temporal motion vector predictor (SbTMVP) candidate associated with the sub-block.
5 . The method of claim 1 , further comprising:
determining, for a second conversion between a second video block of the video and a bitstream of the video using a geometry partition mode in which the second video block is partitioned into at least a third prediction partition and a fourth prediction partition, a final prediction block from a first prediction block and a second prediction block from motion information that is different from a first set of motion information derived for sample positions of the third prediction partition, a second set of motion information derived for sample positions of the fourth prediction partition; and
performing the second conversion based on the final prediction block,
wherein the second video block is coded using a multi-hypothesis technique, and
wherein the second prediction block is uni-predictively or bi-predictively generated using a motion vector or a reference picture different from the first set of motion information and the second set of motion information.
6 . The method of claim 5 , wherein the second prediction block is generated using a third set motion information for the third prediction partition and a fourth set of motion information for the fourth prediction partition, or
wherein the second conversion is further based on an overlapped block motion compensation (OBMC) rule that specifies whether use of OBMC coding is enabled at sample positions of the second video block, and wherein the OBMC rule specifies that at least one of the following:
the OBMC coding is disabled for the second conversion due to the second video block using the geometry partition mode; and
OBMC coding is disabled at sample positions in a weighted area of the second video block and enable OBMC at remaining sample positions.
7 . The method of claim 5 , wherein samples of the final prediction block in the third and fourth prediction partitions excluding a weighted area are generated from the first prediction block or the second prediction block,
wherein, for samples in the weighted area, a weighted prediction from the first prediction block and the second prediction block is used, and
wherein the weighted area crosses a boundary of two prediction partitions.
8 . The method of claim 5 , further comprising determining whether an inheritance of generalized bi-prediction (GBi) weighting factors from motion candidates used for motion information derivation for the prediction partition is enabled at sample positions of the second video block according to an inheritance rule,
wherein the inheritance rule specifies that at least one of the following:
the inheritance is disabled for all positions of the second video block;
the inheritance is enabled for all positions of the second video block; and
the inheritance is disabled for samples positions outside a weighted area of the second video block and enabled at sample positions in the weighted area, wherein the weighted area crosses a boundary of two prediction partitions.
9 . The method of claim 5 , wherein the geometry partition mode is disabled due to the second video block satisfying a size condition, and
wherein the size condition specifies not to use the geometry partition mode due to the first video block having a size greater than a first threshold, or wherein the size condition specifies not to use the geometry partition mode due to the first video block having a size less than a second threshold, or wherein the geometry partition mode is disabled for the first video block due to the first video block having a pre-defined width to height ratio.
10 . The method of claim 1 , further comprising:
performing a third conversion between a third video block of the video and a bitstream of the video in accordance with a rule that defines a format of syntax elements of the bitstream,
wherein the rule specifies whether indication of use of a first coding mode is signaled or whether to signal the indication of use of a first coding mode is based on use of a second coding mode for the third video block,
wherein the first coding mode comprises a geometry partition mode in which the third video block corresponding to multiple prediction partitions, an inter-intra coding mode in which a prediction block of the third video block is derived from an intra prediction signal and an inter prediction signal, and a sub-block merge coding mode in which the third conversion uses derived motion information for each sub-block within the third video block,
wherein the second coding mode is a merge mode, and
wherein the merge mode enables inheriting motion information from a merge candidate in a merge candidate list without a motion vector difference for all of the third video block.
11 . The method of claim 10 , wherein a) the rule specifies that a value of the indication indicative of use of the first coding mode is set to FALSE due to the second coding mode being enabled for the third video block, or that the indication of use of the first coding mode is excluded from the bitstream due to the second coding mode being enabled for the third video block, or
b) the rule includes a first rule that specifies whether syntax elements signaling multiple coding modes are included in the bitstream and/or a second rule that specifies an order in which the syntax elements signaling the multiple coding modes are included in the bitstream, wherein the second rule specifies one of the following orders of syntax elements:
a merge with motion vector difference (MMVD) mode, the sub-block merge mode, the geometry partition mode, and the inter-intra mode; or
the MMVD mode, the sub-block merge mode, the inter-intra mode, and the geometry partition mode; or
the sub-block merge mode is signaled before the MMVD mode; or
the sub-block merge mode, the MMVD mode, the inter-intra mode, the geometry partition mode,
wherein the multiple coding modes include two or more of a geometry partition mode, an inter-intra mode, a sub-block merge mode or a merge with motion vector differencing (MMVD) mode.
12 . The method of claim 11 , wherein c) the order is changed in the bitstream at a video unit level based on a coding condition of the video, and wherein the coding condition comprises a value of a low delay check flag, coding information of previously coded third video blocks, or a dimension of the third video block, or
d) the video unit corresponds to a block, a coding unit, a prediction unit, a coding tree unit or a largest coding unit row, a video slice, a picture or a sequence level.
13 . The method of claim 10 , further comprising:
(i) checking whether a neighboring motion vector precision information of a neighboring block of the third video block is available for determining a motion vector precision for a current video block based on a position of the neighboring block,
wherein, in a case that the neighboring block and the current video block are in different largest coding unit (LCU) rows or in different regions of the video, then the neighboring motion vector precision information is considered to be unavailable; or in a case that the neighboring block and the current video block are in different largest coding unit (LCU) rows, then the motion vector precision for the current video block is determined by assuming the neighboring block to be in a different slice or a different tile, and wherein, due to determining that the neighboring motion vector precision information is unavailable, the motion vector precision for the current video block is set to a default value, and
wherein the current video block and the neighboring block are coded using affine coding mode, or
(ii) determining a context coding mode used for the bitstream based on a position of the neighboring block with respect to a largest coding unit (LCU) row of the current video block,
wherein the neighboring block is considered unavailable due to being in a different LCU row, and wherein the context coding mode uses a single context for coding a syntax element; or the neighboring block is considered unavailable due to being in a different LCU row, and wherein the context coding mode uses bypass coding for coding a syntax element, and wherein the syntax element is one of: alf_ctb_flag, qt_split_cu_flag, mtt_split_cu_flag, mtt_split_cu_flag, cu_skip_flag, amvr_mode, merge_subblock_flag, merge_triangle_flag, inter_affine_flag, cu_skip_flag, pred_mode_flag, pcm_flag, intra_luma_ref_idx, intra_luma_mpm_flag, intra_luma_mpm_idx, intra_luma_mpm_remainder, intra_chroma_pred_mode, merge_flag, inter_pred_idc, inter_affine_flag, cu_affine_type_flag, ref_idx_10, mvp_10_flag, ref_idx_11 mvp_11_flag, amvr_flag, amvr_4pel_flag, gbi_idx, cu_cbf, mmvd_flag, or mmvd_merge_flag.
14 . The method of claim 1 , wherein an indication in the bitstream signals, at a sequence parameter set level or a picture parameter set level or a video parameter set level or a picture header or a slice header or a tile group header or a coding tree unit level, use of the geometry partition mode for the first video block or second video block.
15 . The method of claim 1 , wherein the geometry partition mode includes multiple splitting patterns and each splitting pattern divides the first video block or second video block into two or more partitions, and wherein at least one of the two or more partitions is non-square and non-rectangular or wherein the geometry partition mode comprises a triangular partitioning mode.
16 . The method of claim 1 , wherein the first conversion comprises encoding the video into the bitstream.
17 . The method of claim 1 , wherein the first conversion comprises decoding the bitstream to generate pixel values of the video.
18 . An apparatus for coding video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
make a determination that a first conversion between a first video block of a video region of a video and a bitstream of the video uses a geometry partition mode in which the first video block is partitioned into multiple prediction partitions including at least a first prediction partition;
derive N motion information candidates using one or more sub-block merge candidates associated with the first video block and adding the N motion information candidates to a motion candidate list for the first video block, where N is a positive integer;
derive, from the motion candidate list, motion candidates for the multiple prediction partitions; and
perform the conversion based on the motion candidates for the multiple prediction partitions.
19 . A non-transitory computer-readable storage medium storing instructions that cause a processor to:
make a determination that a first conversion between a first video block of a video region of a video and a bitstream of the video uses a geometry partition mode in which the first video block is partitioned into multiple prediction partitions including at least a first prediction partition;
derive N motion information candidates using one or more sub-block merge candidates associated with the first video block and adding the N motion information candidates to a motion candidate list for the first video block, where N is a positive integer;
derive, from the motion candidate list, motion candidates for the multiple prediction partitions; and
perform the conversion based on the motion candidates for the multiple prediction partitions.
20 . A method for storing a bitstream of a video comprising:
making a determination that generation of a bitstream of a video uses a geometry partition mode in which a first video block of a video region of the video is partitioned into multiple prediction partitions including at least a first prediction partition;
deriving N motion information candidates using one or more sub-block merge candidates associated with the first video block and adding the N motion information candidates to a motion candidate list for the first video block, where N is a positive integer;
deriving, from the motion candidate list, motion candidates for the multiple prediction partitions;
generating the bitstream based on the motion candidates for the multiple prediction partitions; and
storing the bitstream in a non-transitory computer-readable recording medium.