IP Library › Granted Patent US 12,375,705
Granted Patent B2
US 12,375,705 · App. 18/367,431 · Granted Jul 29, 2025

System and method for signaling of motion merge modes in video coding

Inventors: Yi-Wen Chen (San Diego, CA); Xianglin Wang (San Diego, CA)
Assignee: BEIJING DAJIA INTERNET INFORMATION TECHNOLOGY CO., LTD.
H04N19/46H04N19/13H04N19/159H04N19/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,375,705
App. No.
18/367,431
Granted
Jul 29, 2025
Kind
B2
Abstract

The present disclosure relates to a method for video encoding including: signaling a regular merge flag for a coding unit (CU) that is coded as merge mode and merge related modes; when the regular merge flag is signaled as one, indicating that a regular merge mode or merge mode with motion vector differences (MMVD) is used by the CU, constructing a motion vector merge list for the CU and using regular merge index to indicate which candidate is used; and when the regular merge flag is signaled as zero, indicating the regular merge mode is not used by the CU, and further signaling mode flags to indicate associated merge related modes are used when a mode flag's constraints are met; and when the regular merge flag is signaled as one, determining whether a MMVD merge flag is signaled based on the value of a MMVD flag.

Claims (67)

1. A method for video encoding, comprising:

signaling, by an encoder, a regular merge flag for a coding unit (CU) that is coded as merge mode and merge related modes;

in response to determining that the regular merge flag is signaled as one, indicating that a regular merge mode or merge mode with motion vector differences (MMVD) is used by the CU, constructing, by the encoder, a motion vector merge list for the CU and using regular merge index to indicate which candidate is used;

in response to determining that the regular merge flag is signaled as zero, indicating the regular merge mode is not used by the CU, and further signaling, by the encoder, mode flags to indicate a different merge mode is used in response to determining that a mode flag's constraints are met; and

when the regular merge flag is signaled as one, determining whether a MMVD merge flag is signaled based on the value of a MMVD flag, wherein the MMVD flag indicates whether the MMVD is enabled.

2. The method of claim 1 , further comprising:

signaling, by the encoder, the MMVD merge flag in response to determining that the MMVD flag is equal to one.

3. The method of claim 1 , further comprising encoding a subblock flag and the subblock flag's constraints includes:

obtaining, by the encoder, a coding block, wherein the coding block has a width and height;

specifying, by the encoder, whether maximum number of subblock-based merging MVP candidates (MaxNumSubblockMergeCand) is greater than zero;

specifying, by the encoder, whether coding block width is greater than equal to eight; and

specifying, by the encoder, whether coding block height is greater than equal to eight.

4. The method of claim 1 , wherein the mode flag is a combined inter and intra prediction (CIIP) flag and the CIIP flag's constraints include:

obtaining, by the encoder, a coding block, wherein the coding block has a width and height;

specifying, by the encoder, whether sps_mh_intra_enabled_flag is set;

specifying, by the encoder, whether cu_skip_flag is equal to zero;

specifying, by the encoder, whether the coding block width times the coding block height is greater than equal to sixty-four;

specifying, by the encoder, whether coding block width is less than one hundred and twenty-eight; and

specifying, by the encoder, whether coding block height is less than one hundred and twenty-eight.

5. The method of claim 1 , further comprising:

signaling, by the encoder, a regular merge flag before signaling, by the encoder, the mode flag in response to determining that the mode flag's constraints are met.

6. The method of claim 5 , further comprising:

signaling, by the encoder, a regular merge flag before signaling, by the encoder, a merge mode with motion vector differences (MMVD) flag in response to determining that the merge mode with motion vector differences (MMVD) flag's constraints are met.

7. The method of claim 5 , further comprising:

signaling, by the encoder, a regular merge flag before signaling, by the encoder, a combined inter and intra prediction (CIIP) flag in response to determining that the CIIP flag's constraints are met.

8. The method of claim 1 , further comprising:

signaling, by the encoder, a regular merge flag after signaling, by the encoder, a mode flag in response to determining that the mode flag's constraints are met.

9. The method of claim 8 , further comprising:

signaling, by the encoder, a regular merge flag after signaling, by the encoder, a subblock merge mode flag in response to determining that the subblock merge mode flag's constraints are met.

10. The method of claim 1 , further comprising:

signaling, by the encoder, the regular merge flag using Context-adaptive binary arithmetic coding (CABAC) with multiple context models and the selection of the context model is based on coded information.

11. A computing device, comprising:

one or more processors;

a non-transitory computer-readable memory storing instructions executable by the one or more processors, wherein the one or more processors are configured to:

signal, by an encoder, a regular merge flag for a coding unit (CU) that is coded as merge mode and merge related modes;

in response to determining that the regular merge flag is signaled as one, indicate that a regular merge mode or merge mode with motion vector differences (MMVD) is used by the CU, construct, by the encoder, a motion vector merge list for the CU and using regular merge index to indicate which candidate is used;

in response to determining that the regular merge flag is signaled as zero, indicate the regular merge mode is not used by the CU, further signal mode flags to indicate a different merge mode is used in response to determining that a mode flag's constraints are met; and

when the regular merge flag is signaled as one, determine whether a MMVD merge flag is signaled based on the value of a MMVD flag, wherein the MMVD flag indicates whether the MMVD is enabled.

12. The computing device of claim 11 , wherein the one or more processors are configured to:

signal, by the encoder, the MMVD merge flag in response to determining that the MMVD flag is equal to one.

13. The computing device of claim 11 , wherein the one or more processors are further configured to signal a subblock flag and the subblock flag's constraints includes:

obtain, by the encoder, a coding block, wherein the coding block has a width and height;

specify, by the encoder, whether maximum number of subblock-based merging MVP candidates (MaxNumSubblockMergeCand) is greater than zero;

specify, by the encoder, whether coding block width is greater than equal to eight; and

specify, by the encoder, whether coding block height is greater than equal to eight.

14. The computing device of claim 11 , wherein the mode flag is a combined inter and intra prediction (CIIP) flag and the CIIP flag's constraints include:

obtain, by the encoder, a coding block, wherein the coding block has a width and height;

specify, by the encoder, whether sps_mh_intra_enabled_flag is set;

specify, by the encoder, whether cu_skip_flag is equal to zero;

specify, by the encoder, whether the coding block width times the coding block height is greater than equal to sixty-four;

specify, by the encoder, whether coding block width is less than one hundred and twenty-eight; and

specify, by the encoder, whether coding block height is less than one hundred and twenty-eight.

15. The computing device of claim 11 , wherein the one or more processors are configured to:

signal, by the encoder, a regular merge flag before signaling, by the encoder, the mode flag in response to determining that the mode flag's constraints are met.

16. The computing device of claim 15 , wherein the one or more processors are configured to:

signal, by the encoder, a regular merge flag before signaling, by the encoder, a combined inter and intra prediction (CIIP) flag in response to determining that the CIIP flag's constraints are met.

17. The computing device of claim 11 , wherein the one or more processors are configured to:

signal, by the encoder, a regular merge flag after signaling, by the encoder, a mode flag in response to determining that the mode flag's constraints are met.

18. The computing device of claim 17 , wherein the one or more processors are configured to:

signal, by the encoder, a regular merge flag after signaling, by the encoder, a subblock merge mode flag in response to determining that the subblock merge mode flag's constraints are met.

19. The computing device of claim 11 , wherein the one or more processors are configured to:

signal, by the encoder, the regular merge flag using Context-adaptive binary arithmetic coding (CABAC) with multiple context models and the selection of the context model is based on coded information.

20. A non-transitory computer-readable storage medium storing a bitstream generated by an encoding method including the following operations:

signaling, by an encoder, a regular merge flag for a coding unit (CU) that is coded as merge mode and merge related modes;

in response to determining that the regular merge flag is signaled as one, indicating that a regular merge mode or merge mode with motion vector differences (MMVD) is used by the CU, constructing, by the encoder, a motion vector merge list for the CU and using regular merge index to indicate which candidate is used;

in response to determining that the regular merge flag is signaled as zero, indicating the regular merge mode is not used by the CU, and further signaling, by the encoder, mode flags to indicate a different merge mode is used in response to determining that a mode flag's constraints are met; and

when the regular merge flag is signaled as one, determining whether a MMVD merge flag is signaled based on the value of a MMVD flag, wherein the MMVD flag indicates whether the MMVD is enabled.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2023
From: CHEN, YI-WEN; WANG, XIANGLIN
To: BEIJING DAJIA INTERNET INFORMATION TECHNOLOGY CO., LTD.
Reel/Frame 064885/0094 →
Continuity (5)
Continuation 17861111 · Jul 8, 2022
Continuation 17364504 · Jun 30, 2021
Continuation PCTUS2019068977 · Dec 30, 2019
Provisional Application 62787230 · Dec 31, 2018
Related Publication 20230421792A1 · Dec 28, 2023
References Cited (64)
US 9066104B2 · Wang et al. · 2015 [cited by applicant]
US 10182240B2 · Hsu · 2019 [cited by examiner]
US 20120320984A1 · Zhou · 2012 [cited by applicant]
US 20130016785A1 · Wang et al. · 2013 [cited by applicant]
US 20150092851A1 · Yoshikawa et al. · 2015 [cited by applicant]
US 20150264372A1 · Kolesnikov et al. · 2015 [cited by applicant]
US 20160057420A1 · Pang et al. · 2016 [cited by applicant]
US 20170013276A1 · Chen et al. · 2017 [cited by applicant]
US 20170188044A1 · Takehara · 2017 [cited by examiner]
US 20170214932A1 · Huang · 2017 [cited by applicant]
US 20170280159A1 · Xu et al. · 2017 [cited by applicant]
US 20180077417A1 · Huang · 2018 [cited by applicant]
US 20180227582A1 · Park et al. · 2018 [cited by applicant]
US 20180310017A1 · Chen et al. · 2018 [cited by applicant]
US 20180316913A1 · Jun et al. · 2018 [cited by applicant]
US 20180332279A1 · Kang et al. · 2018 [cited by applicant]
US 20210006788A1 · Zhang et al. · 2021 [cited by applicant]
US 20210006790A1 · Zhang · 2021 [cited by examiner]
US 20210144366A1 · Zhang et al. · 2021 [cited by applicant]
US 20210152816A1 · Zhang et al. · 2021 [cited by applicant]
US 20210329251A1 · Chen et al. · 2021 [cited by applicant]
US 20220021900A1 · Jeong et al. · 2022 [cited by applicant]
US 20220116612A1 · Jang · 2022 [cited by examiner]
CN 107113424A · 2017 [cited by applicant]
CN 108886619A · 2018 [cited by applicant]
CN 106797476B · 2019 [cited by examiner]
GB 2579763A · 2020 [cited by applicant]
WO 2020117619A1 · 2020 [cited by applicant]
WO WO2020123998A1 · 2020 [cited by examiner]
Japanese Intellectual Property Office, “Notice of Reasons for Refusal”, issued in Application No. 2023-203270 dated on Jul. 16, 2024, with English Translation, (6p). [cited by applicant]
Japanese Intellectual Property Office, “Notice of Reasons for Refusal”, issued in Application No. 2023-211116 dated on Aug. 6, 2024, with English Translation, (6p). [cited by applicant]
First Office Action issued to Chinese Application No. 202210165633.2 dated Jan. 18, 2023, with English translation (7p). [cited by applicant]
First Office Action issued to Indian Application No. 202248055734 dated Jan. 9, 2023, with partial English translation (7p). [cited by applicant]
Bross, Benjamin, et al., “Versatile Video Coding (Draft 3)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC I/SC 29/WG11, JVET-L1001-v7, 12th Meeting: Macao, China, Oct. 3-12, 2018, (13p). [cited by applicant]
Bross, Benjamin et al., “Versatile Video Coding (Draft 3)”, Joint Video Experts (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L1001-v7, 12th Meeting: Macao, China, Oct. 3-12, 2018, (221p). [cited by applicant]
Sharp Corporation, Tomonori Hashimoto, et al., “CE4: Enhanced Merge with MVD (Test 4.4.4)”., Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11,JVET-M0060, 13th Meeting: Marrakech, MA, Jan… [cited by applicant]
Shen, Yan-Fei et al., “High Efficiency Video Coding”, Chinese Journal of Computers, vol. 36, No. 11, Nov. 2018 (16p). [cited by applicant]
International Search Report and Written Opinion of PCT Application No. PCT/US2019/068977 dated Apr. 23, 2020, (8p). [cited by applicant]
First Office Action of Indian Patent Application 202147029403, dated Mar. 4, 2022 with English translation, (6p). [cited by applicant]
First Office Action of Japanese Patent Application No. 2021-538770, dated Dec. 14, 2021 with English translation, (12p). [cited by applicant]
Rejection Decision of Japanese Patent Application No. 2021-538770, dated Feb. 22, 2022 with English translation, (6p). [cited by applicant]
Bross, Benjamin et al., “Versatile Video Coding (Draft 3)”, Joint Video Experts (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L1001-v7, 12th Meeting: Macao, China, Oct. 3-12, 2018, (13p). [cited by applicant]
Bross, Benjamin et al., “Versatile Video Coding (Draft 6)”, Joint Video Experts (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-O2001-vE, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (113p). [cited by applicant]
Kwai Inc., Yi-Wen Chen, et al., “Non-CE4: Regular merge flag coding”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-M023 I, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, (3p). [cited by applicant]
Huang et al., “Non-CE4: Merge Modes Signaling”, Joint Video Experts (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-00249, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (10p). [cited by applicant]
Supplementary EP Search Report issued to EPApplication No. 19907518.5 dated Feb. 11, 2022, (4p). [cited by applicant]
EP Communication of EP Application No. 19907518.5 dated Feb. 23, 2022, (9p). [cited by applicant]
Office Action issued to Korean Application No. 2021-7020439 dated Mar. 24, 2022 with English translation, (9p). [cited by applicant]
Bross, Benjamin et al., Versatile Video Coding (Draft 3), Joint Video Experts (JVET) of ITU-T SG 16 WP 3 and ISOEC JTC 1/SC 29/WG 11, JVET-L 1001-v7, 12 Meeting: Macao, China, Oct. 3-12, 2018, (223p). [cited by applicant]
Hashimoto, Tomonori, et al., “CE4: Enhanced Merge with MVD (Test 4.4.4)”, Joint Video Experts Team (JVET) of ITU-T-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-M0060_WD, 13 Meeting: Marrakech, MA, Jan. 9-18, 2019, (… [cited by applicant]
Chen, Jianle, et al., “Algorithm description for Versatile Video Coding and Test Model 3 (VTM3)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-L 1002-v1, 12th Meeting: Macao, C… [cited by applicant]
Office Action Notice of Reasons for Refusal issued in JP Application No. 2022-100700 dated Jul. 7, 2023 with English translation, (4p). [cited by applicant]
Bross, Benjamin et al., Versatile Video Coding {Draft 3) Joint Video Experts Team (JVET) of ITU-T SG 16 WP3 and SO/IEC JTC 1/SC 29/WG 11, Document JVET-L 1001-v7, 12th Meeting: Macao, CN Oct. 3-12, 2018, (14p). [cited by applicant]
Chen, Yi-Wen, et al., “Non-CE4: Regular merge flag coding” Joint Video Experts Team (JVET) of ITU-T SG 16 WP3 and ISO/IEC JTC 1/SC 29/WG11, Document: JVET-M0231, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, (5p). [cited by applicant]
Huang, Han et al., “Non-CE4: Merge Modes Signaling” Joint Video Experts Team (JVET) of ITU-T SG 16 WP3 and SO/IEC JTC 1/SC 29/WG11, Document: JVET-O0249, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (12p). [cited by applicant]
Bross, Benjamin et al., Versatile Video Coding {Draft 6) Joint Video Experts Team (JVET) of ITU-T SG 16 WP3 and SO/IEC JTC 1/SC 29/WG 11, Document JVET-O2001-vE, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (8p). [cited by applicant]
Chen, Yi-Wen, et al. “CE4: Regular merge flag coding,” (CE4-1.2.a and CE4-1.2.b), Joint Video Exploration Team (JVET) of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16), JVET-N0324, Mar. 16, 2019, (3p). [cited by applicant]
Ramakrishna et al., “Design Approaches for Mode Decision in HEVC Encoder;” (c) Scitepress, DOI: 10.5220/0004613600230027; Sigmap-2013 (Year: 2013) (5p). [cited by applicant]
European Office Action of EP Application No. 19907518.5 dated Apr. 16, 2024, (7p). [cited by applicant]
Benjamin Bross, et al, “Versatile Video Coding (Draft 3)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Meeting: Macao, China, Oct. 3-12, 2018, (14p). [cited by applicant]
Yi-Wen Chen, et al, “Non-CE4: Regular Merge Flag Coding” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marraketch, MA, Jan. 9-18, 2019, (5p). [cited by applicant]
Han Huang, et al, “Non-CE4: Merge Modes Signaling” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (12p). [cited by applicant]
Benjamin Bross, et al, “Versatile Video Coding (Draft 6)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, (8p). [cited by applicant]
Granting Notice issued in JP application No. 2022-100700, dated on Oct. 31, 2023 with English translation, (4p). [cited by applicant]