Granted Patent B2

US 12,652,381 · App. 18/538,728 · Granted Jun 9, 2026

Inventors: Na Zhang (Beijing, CN); Kai Zhang (Los Angeles, CA); Li Zhang (Los Angeles, CA)

Assignees: BEIJING BYTEDANCE NETWORK TECHNOLOGY CO., LTD.; BYTEDANCE INC.

H04N19/105H04N19/137H04N19/176H04N19/196

Abstract

Claims (68)

1 . A method for video processing, comprising:

determining, during a conversion between a target video block of a video and a bitstream of the video, a matching cost of a target motion candidate for the target video block, the matching cost comprising a template matching cost or a bilateral matching cost;

determining a cost metric for the target motion candidate at least based on the matching cost of the target motion candidate; and

performing the conversion based on a comparison of the cost metric and a further matching cost for the target video block, the further matching cost comprising a further template matching cost or a further bilateral matching cost,

wherein determining the cost metric for the target motion candidate comprises: calculating the cost metric based on a product of the matching cost and a cost factor for the target motion candidate,

wherein calculating the cost metric based on a product of the matching cost and the cost factor comprises one of:

calculating the cost metric by adding a deviation to the product of the matching cost and the cost factor, or

calculating the cost metric by shifting the product of the matching cost and the cost factor by an offset.

2 . The method of claim 1 ,

wherein at least one of the cost factor, the deviation or the offset is an integer.

3 . The method of claim 1 , wherein the template matching cost is based on a difference between samples in a current template for the target video block and samples of a reference template, the reference template being based on the target motion candidate, and

wherein the bilateral matching cost is based on a difference between samples in a first reference frame and samples in a second reference frame for the current video block, the first and second reference frames being based on the target motion candidate.

4 . The method of claim 1 , wherein the cost factor for the target motion candidate depends on a position of the target motion candidate in a candidate list before the candidate list being reordered, or

wherein a first value of the cost factor for the target motion candidate is larger than or smaller than the second value of a further cost factor for a further motion candidate, the target motion candidate being at a first position in a candidate list, the further motion candidate being at a second position behind the first position in the candidate list, or

wherein a first value of the cost factor for the target motion candidate is larger or smaller than the second value of a further cost factor for a further motion candidate, the target motion candidate being in a first group, the further motion candidate being in a second group behind the first group, or

wherein if the target motion candidate comprises one of a subblock motion candidate, an affine motion candidate or a subblock-based temporal motion vector prediction (SbTMVP) motion candidate, the cost metric of the target motion candidate is determined without applying the cost factor.

5 . The method of claim 1 , wherein the cost factor of the target motion candidate in a group or at a position is adaptive, or

wherein the cost factor of the target motion candidate in a group or at a position depends on a coding mode of neighbor coded blocks, or

wherein the neighbor coded blocks comprise at least one of five spatial neighbor blocks and temporal neighbor blocks.

6 . The method of claim 5 , wherein the target motion candidate comprises a subblock-based temporal motion vector prediction (SbTMVP) merge candidate, and the cost factor for the target motion candidate depends on the number of neighbor affine coded blocks, and

wherein:

if the number of spatial neighbor affine coded blocks for the target SbTMVP merge candidate is less than a first threshold number, the cost factor for the target SbTMVP merge candidate is equal to a third value;

if the number of spatial neighbor affine coded blocks for the target SbTMVP merge candidate is equal to the first threshold number, the cost factor for the target SbTMVP merge candidate is equal to a fourth value;

if the number of spatial neighbor affine coded blocks for the target SbTMVP merge candidate is equal to a second threshold number larger than the first threshold number, the cost factor for the target SbTMVP merge candidate is equal to a fifth value; and

if the number of spatial neighbor affine coded blocks for the target SbTMVP merge candidate is larger than the second threshold number, the cost factor for the target SbTMVP merge candidate is equal to the fifth value or a sixth value.

7 . The method of claim 6 , wherein the first threshold number is equal to 1, and the second threshold number is equal to 2; and

wherein:

the third value is equal to 0.2, the fourth value is equal to 0.5, the fifth value is equal to 0.8, and the sixth value is equal to 1; or

the third value is equal to 2, the fourth value is equal to 5, the fifth value is equal to 8, and the sixth value is equal to 10, and a cost factor of affine merge candidates is equal to 10.

8 . The method of claim 1 , further comprising:

comparing the cost metric with the further matching cost of a further motion candidate for the target video block.

9 . The method of claim 1 , wherein a first subgroup size for a first coding mode is different from a second subgroup size for a second coding mode.

10 . The method of claim 9 , wherein if the coding mode comprises a regular merge mode, the subgroup size is a first number,

if the coding mode comprises a subblock merge mode, the subgroup size is a second number, and

if the coding mode comprises a template matching merge mode, the subgroup size is a third number.

11 . The method of claim 10 , wherein the first number is equal to 5, the second number is equal to 3, and the third number is equal to 3.

12 . The method of claim 9 , wherein

if the coding mode comprises a subblock merge mode, the subgroup size is larger than or equal to the maximum number of subblock merge candidates defined in one of: a sequence parameter set (SPS), a picture header or a slice header; and

if the coding mode comprises a template matching merge mode, the subgroup size is larger than or equal to the maximum number of template matching merge candidates defined in one of: the SPS, the picture header or the slice header.

13 . The method of claim 9 , wherein the subgroup size for the coding mode depends on the maximum number of motion candidates in the coding mode, or

wherein the coding mode comprises a subblock merge mode, and the subgroup size depends on the number of neighbor affine coded blocks.

14 . The method of claim 13 , wherein the neighbor affine coded blocks comprise at least one of five spatial neighbor blocks and temporal neighbor blocks, or

wherein if the number of spatial neighbor affine coded blocks is less than or equal to a threshold size, the subgroup size is a first size, and

if the number of spatial neighbor affine coded blocks is larger than the threshold size, the subgroup size is a second size larger than or smaller than the first size.

15 . The method of claim 14 , wherein the threshold size is equal to 1, the first size is equal to 3, and the second size is equal to 5.

16 . The method of claim 9 , wherein the coding mode comprises one of:

a regular merge mode,

a subblock merge mode, or

a template matching merge mode.

17 . The method of claim 1 , further comprising:

storing the bitstream in a non-transitory computer-readable recording medium.

18 . The method of claim 1 , wherein the conversion includes encoding the target video block into the bitstream, or decoding the target video block from the bitstream.

19 . An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to perform a method comprising:

determining, during a conversion between a target video block of a video and a bitstream of the video, a matching cost of a target motion candidate for the target video block, the matching cost comprising a template matching cost or a bilateral matching cost;

determining a cost metric for the target motion candidate at least based on the matching cost of the target motion candidate; and

performing the conversion based on a comparison of the cost metric and a further matching cost for the target video block, the further matching cost comprising a further template matching cost or a further bilateral matching cost,

wherein determining the cost metric for the target motion candidate comprises: calculating the cost metric based on a product of the matching cost and a cost factor for the target motion candidate,

wherein calculating the cost metric based on a product of the matching cost and the cost factor comprises one of:

calculating the cost metric by adding a deviation to the product of the matching cost and the cost factor, or

calculating the cost metric by shifting the product of the matching cost and the cost factor by an offset.

20 . A non-transitory computer-readable recording medium storing program instructions and a bitstream of a video which is generated by a processor executing the program instructions to perform a method wherein the method comprises:

determining, during a conversion between a target video block of a video and a bitstream of the video, a matching cost of a target motion candidate for the target video block, the matching cost comprising a template matching cost or a bilateral matching cost;

determining a cost metric for the target motion candidate at least based on the matching cost of the target motion candidate; and

generating the bitstream based on a comparison of the cost metric and a further matching cost for the target video block, the further matching cost comprising a further template matching cost or a further bilateral matching cost,

wherein determining the cost metric for the target motion candidate comprises: calculating the cost metric based on a product of the matching cost and a cost factor for the target motion candidate,

wherein calculating the cost metric based on a product of the matching cost and the cost factor comprises one of:

calculating the cost metric by adding a deviation to the product of the matching cost and the cost factor, or

calculating the cost metric by shifting the product of the matching cost and the cost factor by an offset.

Assignments (3)

Priority Claims (1)

Continuity (2)

References Cited (19)