IP Library › Granted Patent US 12,363,334
Granted Patent B2
US 12,363,334 · App. 18/242,486 · Granted Jul 15, 2025

System and method for signaling of motion merge modes in video coding

Inventors: Yi-Wen Chen (San Diego, CA); Xianglin Wang (San Diego, CA)
Assignee: BEIJING DAJIA INTERNET INFORMATION TECHNOLOGY CO., LTD.
H04N19/46H04N19/13H04N19/159H04N19/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,363,334
App. No.
18/242,486
Granted
Jul 15, 2025
Kind
B2
Abstract

The present disclosure relates to a decoding method. The method includes acquiring, from a decoder, a regular merge flag for a coding unit (CU) that is coded as merge mode and merge related modes; when the regular merge flag is one, indicating that a regular merge mode or merge mode with motion vector differences (MMVD) is used by the CU, constructing a single merge list for the CU and using regular merge index to indicate which candidate is used; when the regular merge flag is zero, indicating the regular merge mode is not used by the CU, and further receiving mode flags to indicate associated merge related modes are used when a mode flag's constraints are met; and when the regular merge flag is one, determining whether a MMVD merge flag is received based on the value of a MMVD flag.

Claims (67)

1. A computer implemented method for motion merge modes, comprising:

acquiring, from a decoder, a regular merge flag for a coding unit (CU) that is coded as merge mode and merge related modes;

when the regular merge flag is one, indicating that a regular merge mode or merge mode with motion vector differences (MMVD) is used by the CU, constructing a single merge list for the CU, wherein the single merge list comprises regular motion vector candidates and MMVD motion vector candidates selected by a regular merge index to indicate which candidate is used, wherein the single merge list is constructed for both the regular merge mode and the MMVD;

when the regular merge flag is zero, indicating the regular merge mode is not used by the CU, and further receiving mode flags to indicate associated merge related modes are used when a mode flag's constraints are met; and

when the regular merge flag is one, determining whether a MMVD merge flag is received based on the value of a MMVD flag, wherein the MMVD flag indicates whether the MMVD is enabled.

2. The method of claim 1 , further comprising:

receiving, by the decoder, the MMVD merge flag when the MMVD flag is equal to one.

3. The method of claim 1 , further comprising receiving subblock flag and the subblock flag's constraints includes:

acquiring a coding block from a decoder, wherein the coding block has a width and height;

determining, by the decoder, whether maximum number of subblock-based merging MVP candidates (MaxNumSubblockMergeCand) is greater than zero;

determining, by the decoder, whether coding block width is greater than equal to eight; and

determining, by the decoder, whether coding block height is greater than equal to eight.

4. The method of claim 1 , wherein the mode flag is a combined inter and intra prediction (CIIP) flag and the CIIP flag's constraints include:

acquiring a coding block from a decoder, wherein the coding block has a width and height;

determining, by the decoder, whether sps_mh_intra_enabled_flag is set;

determining, by the decoder, whether cu_skip_flag is equal to zero;

determining, by the decoder, whether the coding block width times the coding block height is greater than equal to sixty-four;

determining, by the decoder, whether coding block width is less than one hundred and twenty-eight; and

determining, by the decoder, whether coding block height is less than one hundred and twenty-eight.

5. The method of claim 1 , further comprising:

receiving, by the decoder, a regular merge flag before receiving, by the decoder, the mode flag when the mode flag's constraints are met.

6. The method of claim 5 , further comprising:

receiving, by the decoder, a regular merge flag before receiving, by the decoder, a merge mode with motion vector differences (MMVD) flag when the merge mode with motion vector differences (MMVD) flag's constraints are met.

7. The method of claim 5 , further comprising:

receiving, by the decoder, a regular merge flag before receiving, by the decoder, a combined inter and intra prediction (CIIP) flag when the CIIP flag's constraints are met.

8. The method of claim 1 , further comprising:

receiving, by the decoder, a regular merge flag after receiving, by the decoder, a mode flag when the mode flag's constraints are met.

9. The method of claim 8 , further comprising:

receiving, by the decoder, a regular merge flag after receiving, by the decoder, a subblock merge mode flag when the subblock merge mode flag's constraints are met.

10. The method of claim 1 , further comprising:

receiving, by the decoder, the regular merge flag using Context-adaptive binary arithmetic coding (CABAC) with multiple context models and the selection of the context model is based on coded information.

11. A computing device comprising:

one or more processors;

a non-transitory computer-readable memory storing instructions executable by the one or more processors, wherein the one or more processors are configured to:

acquire, from a decoder, a regular merge flag for a coding unit (CU) that is coded as merge mode and merge related modes;

when the regular merge flag is one, indicate that a regular merge mode or merge mode with motion vector differences (MMVD) is used by the CU, construct a single merge list for the CU, wherein the single merge list comprises regular motion vector candidates and MMVD motion vector candidates selected by a regular merge index to indicate which candidate is used, wherein the single merge list is constructed for both the regular merge mode and the MMVD;

when the regular merge flag is zero, indicate the regular merge mode is not used by the CU, and further receive mode flags to indicate associated merge related modes are used when a mode flag's constraints are met; and

when the regular merge flag is one, determine whether a MMVD merge flag is received based on the value of a MMVD flag, wherein the MMVD flag indicates whether the MMVD is enabled.

12. The computing device of claim 11 , further comprising:

receive, by the decoder, the MMVD merge flag when MMVD flag is equal to one.

13. The computing device of claim 11 , wherein the one or more processors are further configured to receive subblock flag and the subblock flag's constraints includes:

acquire a coding block from a decoder, wherein the coding block has a width and height;

determine, by the decoder, whether maximum number of subblock-based merging MVP candidates (MaxNumSubblockMergeCand) is greater than zero;

determine, by the decoder, whether coding block width is greater than equal to eight; and

determine, by the decoder, whether coding block height is greater than equal to eight.

14. The computing device of claim 11 , wherein the mode flag is a combined inter and intra prediction (CIIP) flag and the CIIP flag's constraints include:

acquire a coding block from a decoder, wherein the coding block has a width and height;

determine, by the decoder, whether sps_mh_intra_enabled_flag is set;

determine, by the decoder, whether cu_skip_flag is equal to zero;

determine, by the decoder, whether the coding block width times the coding block height is greater than equal to sixty-four;

determine, by the decoder, whether coding block width is less than one hundred and twenty-eight; and

determine, by the decoder, whether coding block height is less than one hundred and twenty-eight.

15. The computing device of claim 11 , further comprising:

signal, by the decoder, a regular merge flag before signaling, by the decoder, the mode flag when the mode flag's constraints are met.

16. The computing device of claim 15 , further comprising:

signal, by the decoder, a regular merge flag before signaling, by the decoder, a combined inter and intra prediction (CIIP) flag when the CIIP flag's constraints are met.

17. The computing device of claim 11 , further comprising:

receive, by the decoder, a regular merge flag after receiving, by the decoder, a mode flag when the mode flag's constraints are met.

18. The computing device of claim 17 , further comprising:

receive, by the decoder, a regular merge flag after receiving, by the decoder, a subblock merge mode flag when the subblock merge mode flag's constraints are met.

19. The computing device of claim 11 , further comprising:

receive, by the decoder, the regular merge flag using Context-adaptive binary arithmetic coding (CABAC) with multiple context models and the selection of the context model is based on coded information.

20. A non-transitory computer-readable storage medium storing a bitstream to be decoded by a decoding method including the following operations:

acquiring, from a decoder, a regular merge flag for a coding unit (CU) that is coded as merge mode and merge related modes;

when the regular merge flag is one, indicating that a regular merge mode or merge mode with motion vector differences (MMVD) is used by the CU, constructing a single merge list for the CU, wherein the single merge list comprises regular motion vector candidates and MMVD motion vector candidates selected by a regular merge index to indicate which candidate is used, wherein the single merge list is constructed for both the regular merge mode and the MMVD;

when the regular merge flag is zero, indicating the regular merge mode is not used by the CU, and further receiving mode flags to indicate associated merge related modes are used when a mode flag's constraints are met; and

when the regular merge flag is one, determining whether a MMVD merge flag is received based on the value of a MMVD flag, wherein the MMVD flag indicates whether the MMVD is enabled.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE CONVEYING PARTY DATA PREVIOUSLY RECORDED AT REEL: 064811 FRAME: 0061. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Sep 18, 2023
From: CHEN, YI-WEN; WANG, XIANGLIN
To: BEIJING DAJIA INTERNET INFORMATION TECHNOLOGY CO., LTD.
Reel/Frame 064935/0261 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 6, 2023
From: CHEN, YI-WEN; W, XIANGLIN
To: BEIJING DAJIA INTERNET INFORMATION TECHNOLOGY CO., LTD.
Reel/Frame 064811/0061 →
Continuity (5)
Continuation 17861111 · Jul 8, 2022
Continuation 17364504 · Jun 30, 2021
Continuation PCTUS2019068977 · Dec 30, 2019
Provisional Application 62787230 · Dec 31, 2018
Related Publication 20230421791A1 · Dec 28, 2023
References Cited (69)
US 9066104B2 · Wang et al. · 2015 [cited by applicant]
US 10182240B2 · Hsu · 2019 [cited by examiner]
US 20120320984A1 · Zhou · 2012 [cited by examiner]
US 20130016785A1 · Wang et al. · 2013 [cited by applicant]
US 20150092851A1 · Yoshikawa et al. · 2015 [cited by applicant]
US 20150264372A1 · Kolesnikov · 2015 [cited by examiner]
US 20160057420A1 · Pang · 2016 [cited by examiner]
US 20170013276A1 · Chen · 2017 [cited by examiner]
US 20170188044A1 · Takehara · 2017 [cited by examiner]
US 20170214932A1 · Huang · 2017 [cited by applicant]
US 20170280159A1 · Xu · 2017 [cited by examiner]
US 20180077417A1 · Huang · 2018 [cited by applicant]
US 20180227582A1 · Park et al. · 2018 [cited by applicant]
US 20180310017A1 · Chen et al. · 2018 [cited by applicant]
US 20180316913A1 · Jun · 2018 [cited by examiner]
US 20180332279A1 · Kang · 2018 [cited by examiner]
US 20210006788A1 · Zhang · 2021 [cited by examiner]
US 20210006790A1 · Zhang · 2021 [cited by examiner]
US 20210144366A1 · Zhang · 2021 [cited by examiner]
US 20210152816A1 · Zhang · 2021 [cited by examiner]
US 20210329251A1 · Chen et al. · 2021 [cited by applicant]
US 20220021900A1 · Jeong · 2022 [cited by examiner]
CN 107113424A · 2017 [cited by applicant]
CN 108886619A · 2018 [cited by applicant]
CN 106797476B · 2019 [cited by applicant]
GB 2579763A · 2020 [cited by applicant]
WO 2020117619A1 · 2020 [cited by applicant]
WO 2020123998A1 · 2020 [cited by applicant]
Benjamin Bross et al., “Versatile Video Coding (Draft 3)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L1001-v1, 12th Meeting: Macao, CN, Oct. 3-12, 2018, (160p) (Year: 2018). [cited by examiner]
First Office Action issued to Chinese Application No. 202210165633.2 dated Jan. 18, 2023, with English translation (7p). [cited by applicant]
First Office Action issued to Indian Application No. 202248055734 dated Jan. 9, 2023, with partial English translation (7p). [cited by applicant]
Bross, Benjamin, et al., “Versatile Video Coding (Draft 3)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC I/SC 29/WG11, JVET-L1001-v7, 12th Meeting: Macao, China, Oct. 3-12, 2018, (13p). [cited by applicant]
Bross, Benjamin et al., “Versatile Video Coding (Draft 3)”, Joint Video Experts (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L1001-v7, 12th Meeting: Macao, China, Oct. 3-12, 2018, (221p). [cited by applicant]
Sharp Corporation, Tomonori Hashimoto, et al., “CE4: Enhanced Merge with MVD (Test 4.4.4)”., Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11,JVET-M0060, 13th Meeting: Marrakech, MA, Jan… [cited by applicant]
Shen, Yan-Fei et al., “High Efficiency Video Coding”, Chinese Journal of Computers, vol. 36, No. 11, Nov. 2018 (16p). [cited by applicant]
International Search Report and Written Opinion of PCT Application No. PCT/US2019?068977 dated Arpil 23, 2020, (8p). [cited by applicant]
First Office Action of Indian Patent Application 202147029403, dated Mar. 4, 2022 with English translation, (6p). [cited by applicant]
First Office Action of Japanese Patent Application No. 2021-538770, dated Dec. 14, 2021 with English translation, (12p). [cited by applicant]
Rejection Decision of Japanese Patent Application No. 2021-538770, dated Feb. 22, 2022 with English translation, (6p). [cited by applicant]
Bross, Benjamin et al., “Versatile Video Coding (Draft 3)”, Joint Video Experts (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L1001-v7, 12th Meeting: Macao, China, Oct. 3-12, 2018, (13p). [cited by applicant]
Bross, Benjamin et al., “Versatile Video Coding (Draft 6)”, Joint Video Experts (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-O2001-vE, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (113p). [cited by applicant]
Kwai Inc.,Yi-Wen Chen, et al.,“Non-CE4: Regular merge flag coding”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-M023 I, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, (3p). [cited by applicant]
Huang et al., “Non-CE4: Merge Modes Signaling”, Joint Video Experts (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-O0249, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (10p). [cited by applicant]
Supplementary EP Search Report issued to EPApplication No. 19907518.5 dated Feb. 11, 2022, (4p). [cited by applicant]
EP Communication of EP Application No. 19907518.5 dated Feb. 23, 2022, (9p). [cited by applicant]
Office Action issued to Korean Application No. 2021-7020439 dated Mar. 24, 2022 with English translation, (9p). [cited by applicant]
Bross, Benjamin et al., 'Versatile Video Coding (Draft 3), Joint Video Experts (JVET) of ITU-T SG 16 WP 3 and ISOEC JTC 1/SC 29/WG 11, JVET-L 1001-v7, 12 Meeting: Macao, China, Oct. 3-12, 2018, (223p). [cited by applicant]
Hashimoto, Tomonori, et al., “CE4: Enhanced Merge with MVD (Test 4.4.4)”, Joint Video Experts Team (JVET) of ITU-T-T SG 16 WP 3 and ISO/EC JTC 1/SC 29/WG 11, JVET-M0060_WD, 13 Meeting: Marrakech, MA, Jan. 9-18, 2019, (1… [cited by applicant]
Chen, Jianle, et al., “Algorithm description for Versatile Video Coding and Test Model 3 (VTM3)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L 1002-v1, 12th Meeting: Macao, C… [cited by applicant]
Office Action Notice of Reasons for Refusal issued in JP Application No. 2022-100700 dated Jul. 7, 2023 with English translation, (4p). [cited by applicant]
Bross, Benjamin et al., 'Versatile Video Coding {Draft 3) Joint Video Experts Team (JVET) of ITU-T SG 16 WP3 and SO/IEC JTC 1/SC 29/WG 11, Document JVET-L 1001-v7, 12th Meeting: Macao, CN Oct. 3-12, 2018, (14p). [cited by applicant]
Chen, Yi-Wen, et al., “Non-CE4: Regular merge flag coding” Joint Video Experts Team (JVET) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC 29/WG11, Document: JVET-M0231, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, (5p). [cited by applicant]
Huang, Han et al., “Non-CE4: Merge Modes Signaling” Joint Video Experts Team (JVET) of ITU-T SG 16 WP3 and SO/IEC JTC 1/SC 29/WG11, Document: JVET-00249, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (12p). [cited by applicant]
Bross, Benjamin et al., 'Versatile Video Coding {Draft 6) Joint Video Experts Team (JVET) of ITU-T SG 16 WP3 and SO/IEC JTC 1/SC 29/WG 11, Document JVET-O2001-vE, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (8p). [cited by applicant]
Chen, Yi-Wen, et al. “CE4: Regular merge flag coding,” (CE4-1.2.a and CE4-1.2.b), Joint Video Exploration Team (JVET) of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16), JVET-N0324, Mar. 16, 2019, (3p). [cited by applicant]
Ramakrishna et al., “Design Approaches for Mode Decision in HEVC Encoder;” (c) Scitepress, DOI: 10.5220/0004613600230027; Sigmap-2013 (Year: 2013) (5p). [cited by applicant]
Benjamin Bross, et al, “Versatile Video Coding (Draft 3)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Meeting: Macao, China, Oct. 3-12, 2018, (14p). [cited by applicant]
Yi-Wen Chen, et al, “Non-CE4: Regular Merge Flag Coding” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marraketch, MA, Jan. 9-18, 2019, (5p). [cited by applicant]
Han Huang, et al, “Non-CE4: Merge Modes Signaling” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (12p). [cited by applicant]
Benjamin Bross, et al, “Versatile Video Coding (Draft 6)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (8p). [cited by applicant]
Granting Notice issued in JP application No. 2022-100700, dated on Oct. 31, 2023 with English translation, (4p). [cited by applicant]
European Office Action of EP Application No. 19907518.5 dated Apr. 16, 2024, (7p). [cited by applicant]
Japanese Intellectual Property Office, “Notice of Reasons for Refusal”, issued in Application No. 2023-203270 dated on Jul. 16, 2024, with English Translation, (6p). [cited by applicant]
Japanese Intellectual Property Office, “Notice of Reasons for Refusal”, issued in Application No. 2023-211116 dated on Aug. 6, 2024, with English Translation, (6p). [cited by applicant]
CNOA issued in Application No. 202410636938.6 dated Nov. 21, 2024 with English translation, (30p). [cited by applicant]
CNOA issued in Application No. 202410776495.0 dated Nov. 7, 2024 with English translation, (26p). [cited by applicant]
JPOA issued in Application No. 2023-211116 dated Nov. 26, 2024 with English translation, (8p). [cited by applicant]
JPOA issued in Application No. 2023-203270 dated Nov. 26, 2024 with English translation, (8p). [cited by applicant]
Benjamin Bross et al., “Versatile Video Coding (Draft 3)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L1001-v1, 12th Meeting: Macao, CN, Oct. 3-12, 2018, (160p). [cited by applicant]