Granted Patent B2

US 12,401,796 · App. 17/984,948 · Granted Aug 26, 2025

Inventors: Lien-Fei Chen (Hsinchu, TW); Guichun Li (San Jose, CA); Xin Zhao (San Jose, CA); Shan Liu (San Jose, CA)

Assignee: TENCENT AMERICA LLC

H04N19/137H04N19/132H04N19/159H04N19/176

Abstract

Claims (86)

1. A method of video decoding in a decoder, comprising:

receiving a coded bitstream comprising a current block in a current picture, wherein the current block includes a plurality of subblocks;

obtaining prediction information indicating the current block is coded in a subblock-based temporal motion vector prediction (SbTMVP) mode, wherein a subblock in the plurality of subblocks of the current block is bi-predicted;

determining motion information of the subblock based on the SbTMVP mode;

applying at least one of (i) a bilateral matching (BM)-based motion vector (MV) refinement or (ii) a sample-based bi-directional optical flow (BDOF) mode to the subblock to refine the motion information of the subblock, wherein the BM-based MV refinement includes a decoder-side motion vector refinement (DMVR) or a multi-pass decoder-side motion vector refinement (MP-DMVR)), and the BDOF mode is applied after applying the BM-based MV refinement when the BM-based MV refinement and the BDOF mode are applied; and

reconstructing the current block based on the refined motion information of the subblock in the plurality of subblocks.

2. The method of claim 1 , wherein

the applying includes applying the BM-based MV refinement to determine the refined motion information of the subblock; and

the reconstructing includes reconstructing the subblock based on the refined motion information.

3. The method of claim 2 , wherein

the motion information includes an initial MV pair of the subblock;

the BM-based MV refinement includes the DMVR;

the applying includes applying the DMVR to an area in the subblock to determine a refined MV pair of the area based on the initial MV pair, the area being less than or equal to an area of the subblock; and

the reconstructing includes reconstructing the area in the subblock based on the refined MV pair.

4. The method of claim 2 , wherein

the motion information includes an initial MV pair of the subblock;

the BM-based MV refinement includes the MP-DMVR; and

when a subblock size of the subblock is larger than a first threshold M 1 ×N 1 ,

applying at least one DMVR to the subblock to determine a first refined MV pair of the subblock; and

applying the BDOF mode to an area in the subblock to determine a second refined MV pair of the area based on the first refined MV pair, the area being smaller than or equal to an area of the subblock.

5. The method of claim 1 , wherein

the motion information includes an initial MV pair of the subblock;

the applying includes applying the BDOF mode to each sample in the subblock to determine a refined MV pair of the respective sample; and

the reconstructing includes reconstructing each sample in the subblock based on the refined MV pair of the respective sample.

6. The method of claim 2 , wherein the refined motion information of the subblock includes one or more first refined MV pairs of respective one or more areas in the subblock;

a refined MV pair of each sample in an area in the one or more areas is determined based on the BDOF mode and the first refined MV pair corresponding to the area; and

the reconstructing includes reconstructing each sample in the area based on the refined MV pair of the respective sample.

7. The method of claim 1 , wherein the prediction information indicates that the at least one of (i) the BM-based MV refinement or (ii) the BDOF mode is applied to the subblock.

8. The method of claim 1 , wherein the prediction information includes a flag that indicates the at least one of (i) the BM-based MV refinement or (ii) the BDOF mode is applied to the subblock.

9. The method of claim 1 , wherein

the BM-based MV refinement or the BDOF mode is applied based on a first reference picture and a second reference picture of the current picture;

the first reference picture is prior to the current picture in a display order, and the second reference picture is after the current picture in the display order; and

distances from the first reference picture and the second reference picture to the current picture are identical.

10. A method for video encoding in a video encoder, the method comprising:

determining a current block in a current picture is to be coded in a subblock-based temporal motion vector prediction (SbTMVP) mode, wherein a subblock in a plurality of subblocks of the current block is bi-predicted;

determining motion information of the subblock based on the SbTMVP mode;

applying at least one of (i) a bilateral matching (BM)-based motion vector (MV) refinement or (ii) a sample-based bi-directional optical flow (BDOF) mode to the subblock to refine the motion information of the subblock, wherein the BM-based MV refinement includes a decoder-side motion vector refinement (DMVR) or a multi-pass decoder-side motion vector refinement (MP-DMVR), and the BDOF mode is applied after applying the BM-based MV refinement when the BM-based MV refinement and the BDOF mode are applied; and

encoding the current block in a bitstream based on the refined motion information of the subblock in the plurality of subblocks.

11. The method of claim 10 , wherein

the applying includes applying the BM-based MV refinement to determine the refined motion information of the subblock; and

the encoding includes encoding the subblock based on the refined motion information.

12. The method of claim 11 , wherein

the refined motion information of the subblock includes one or more first refined MV pairs of respective one or more areas in the subblock;

a refined MV pair of each sample in an area in the one or more areas is determined based on the BDOF mode and the first refined MV pair corresponding to the area; and

the encoding includes encoding each sample in the area based on the refined MV pair of the respective sample.

13. The method of claim 11 , wherein

the motion information includes an initial MV pair of the subblock;

the BM-based MV refinement includes the DMVR;

the applying includes applying the DMVR to an area in the subblock to determine a refined MV pair of the area based on the initial MV pair, the area being less than or equal to an area of the subblock; and

the encoding includes encoding the area in the subblock based on the refined MV pair.

14. The method of claim 11 , wherein

the motion information includes an initial MV pair of the subblock;

the BM-based MV refinement includes the MP-DMVR; and

when a subblock size of the subblock is larger than a first threshold M 1 ×N 1 ,

applying at least one DMVR to the subblock to determine a first refined MV pair of the subblock; and

applying the BDOF mode to an area in the subblock to determine a second refined MV pair of the area based on the first refined MV pair, the area being smaller than or equal to an area of the subblock.

15. A method of processing video data, the method comprising:

processing a bitstream of the video data, the bitstream comprising a current block in a current picture, wherein the current block includes a plurality of subblocks and the bitstream causes a decoder to

obtain prediction information indicating a current block is coded in a subblock-based temporal motion vector prediction (SbTMVP) mode, wherein a subblock in the plurality of subblocks of the current block is bi-predicted;

determine motion information of the subblock based on the SbTMVP mode;

apply at least one of (i) a bilateral matching (BM)-based motion vector (MV) refinement or (ii) a sample-based bi-directional optical flow (BDOF) mode to the subblock to refine the motion information of the subblock, wherein the BM-based MV refinement includes a decoder-side motion vector refinement (DMVR) or a multi-pass decoder-side motion vector refinement (MP-DMVR)), and the BDOF mode is applied after applying the BM-based MV refinement when the BM-based MV refinement and the BDOF mode are applied; and

reconstruct the current block based on the refined motion information of the subblock in the plurality of subblocks.

16. The method of claim 15 , wherein the bitstream causes the decoder to:

apply the BM-based MV refinement to determine the refined motion information of the subblock; and

reconstruct the subblock based on the refined motion information.

17. The method of claim 15 , wherein

the motion information includes an initial MV pair of the subblock;

the BM-based MV refinement includes the DMVR;

the bitstream causes the decoder to:

apply the DMVR to an area in the subblock to determine a refined MV pair of the area based on the initial MV pair, the area being less than or equal to an area of the subblock; and

reconstruct the area in the subblock based on the refined MV pair.

18. The method of claim 15 , wherein

the motion information includes an initial MV pair of the subblock;

the BM-based MV refinement includes the MP-DMVR; and

when a subblock size of the subblock is larger than a first threshold M 1 ×N 1 , the bitstream causes the decoder to:

apply at least one DMVR to the subblock to determine a first refined MV pair of the subblock; and

apply the BDOF mode to an area in the subblock to determine a second refined MV pair of the area based on the first refined MV pair, the area being smaller than or equal to an area of the subblock.

19. The method of claim 15 , wherein

the motion information includes an initial MV pair of the subblock; and

the bitstream causes the decoder to:

apply the BDOF mode to each sample in the subblock to determine a refined MV pair of the respective sample, and

reconstruct each sample in the subblock based on the refined MV pair of the respective sample.

20. The method of claim 15 , wherein

the refined motion information of the subblock includes one or more first refined MV pairs of respective one or more areas in the subblock;

a refined MV pair of each sample in an area in the one or more areas is determined based on the BDOF mode and the first refined MV pair corresponding to the area; and

the bitstream causes the decoder to reconstruct each sample in the area based on the refined MV pair of the respective sample.

Assignments (1)

Continuity (2)

References Cited (16)