IP Library › Granted Patent US 12,316,877
Granted Patent B2
US 12,316,877 · App. 18/338,886 · Granted May 27, 2025

High-level syntax control flags for template matching-related coding tools in video coding

Inventors: Chun-Chi Chen (San Diego, CA); Han Huang (San Diego, CA); Vadim Seregin (San Diego, CA); Marta Karczewicz (San Diego, CA)
Assignee: QUALCOMM INCORPORATED
H04N19/70H04N19/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,316,877
App. No.
18/338,886
Granted
May 27, 2025
Kind
B2
Abstract

A device for decoding video data comprises one or more processors configured to: obtain a syntax element from a bitstream that includes an encoded representation of the video data; determine, based on the syntax element, that a template-matching tool is enabled; based on the template-matching tool being enabled, applying the template-matching tool to generate a prediction block for a current coding unit (CU) of the video data; and reconstruct the current CU based on the prediction block for the current CU.

Claims (144)

1. A device for decoding video data, the device comprising:

a memory comprising one or more storage media, the memory configured to store the video data; and

one or more processors implemented in circuitry, the one or more processors configured to:

obtain at least one syntax element from a bitstream that includes an encoded representation of the video data;

determine, based on the at least one syntax element, that a template-matching tool is enabled;

based on the at least one syntax element indicating intra-block copy template matching merge mode (IBC-TM-MRG) or intra-block copy template matching advance motion vector prediction (IBC-TM-AMVP) is disabled, set a pruning threshold to a value;

prune a first candidate from a list based on a difference between a motion vector component of the first candidate and a motion vector component of a second candidate in the list being less than the pruning threshold;

based on the template-matching tool being enabled, apply the template-matching tool to generate a prediction block for a current coding unit (CU) of the video data based on a candidate in the list; and

reconstruct the current CU based on the prediction block for the current CU.

2. The device of claim 1 , wherein the template-matching tool is one of:

template-matching advanced motion vector prediction (TM-AMVP),

geometric partitioning mode (GPM) split mode reordering,

candidate reordering for regular merge mode with motion vector difference (MMVD) and affine MMVD,

motion vector difference (MVD) sign prediction,

reference picture reordering,

template-matching merge mode,

template-matching GPM,

template-matching combined inter-intra prediction (CIIP),

adaptive re-ordering of merge candidates,

temporal motion vector prediction (TMVP) and non-adjacent merge candidate type reordering,

template-matching overlapped block motion compensation (TM-OBMC),

intra template matching prediction (IntraTMP),

intra-block copy-template matching-advanced motion vector prediction (IBC-TM-AMVP), or

intra-block copy-template matching-merge (IBC-TM-AMVP).

3. The device of claim 1 , wherein:

the template-matching tool is a first template-matching tool,

the at least one syntax element is a first syntax element, and

the one or more processors are further configured to determine, based on the first syntax element, whether the bitstream includes a second syntax element that indicates whether a second template-matching tool is enabled.

4. The device of claim 3 , wherein the first template-matching tool is template-matching merge mode and the second template-matching tool is template-matching combined intra/inter prediction (CIIP) mode.

5. The device of claim 3 , wherein:

the first template-matching tool is adaptive reordering of merge candidates (ARMC), and

the second template-matching tool is one of: geometric partitioning mode (GPM) split mode reordering, candidate reordering for regular merge mode with motion vector difference (MMVD) and affine MMVD, motion vector difference (MVD) sign prediction, reference picture reordering, temporal motion vector prediction (TMVP) or non-adjacent merge candidate type reordering.

6. The device of claim 1 , wherein:

the at least one syntax element is a first syntax element,

the one or more processors are further configured to:

obtain a second syntax element from the bitstream, wherein the second syntax element is a sequence-level syntax element, a picture-level syntax element, a slice-level syntax element, or a tile-level syntax element; and

determine, based on the second syntax element, whether the bitstream includes one or more template-matching tool syntax elements indicating whether one or more template-matching tools are enabled, wherein the one or more template-matching tool syntax elements include the first syntax element.

7. The device of claim 6 , wherein presence of syntax elements indicating whether intra template matching prediction (intraTMP), intra block copy template matching with advanced motion vector prediction (IBC-TM-AMVP), and intra block copy template matching with merge mode (IBC-TM-AMVP) are enabled is not dependent on the second syntax element.

8. The device of claim 1 , further comprising a display configured to display decoded video data.

9. A device for encoding video data, the device comprising:

a memory comprising one or more storage media, the memory configured to store the video data; and

one or more processors implemented in circuitry, the one or more processors configured to:

based on intra-block copy template matching merge mode (IBC-TM-MRG) or intra-block copy template matching advance motion vector prediction (IBC-TM-AMVP) being disabled, set a pruning threshold to a value;

prune a first candidate from a list based on a difference between a motion vector component of the first candidate and a motion vector component of a second candidate in the list being less than the pruning threshold;

based on a template-matching tool being enabled, apply the template-matching tool to generate a prediction block for a current coding unit (CU) of the video data based on a candidate in the list;

encode the current CU based on the prediction block for the current CU;

signal, in a bitstream that includes an encoded representation of the video data, at least one syntax element that indicates that the template-matching tool is enabled.

10. The device of claim 9 , wherein the template-matching tool is one of:

template-matching advanced motion vector prediction (TM-AMVP),

geometric partitioning mode (GPM) split mode reordering,

candidate reordering for regular merge mode with motion vector difference (MMVD) and affine MMVD,

motion vector difference (MVD) sign prediction,

reference picture reordering,

template-matching merge mode,

template-matching GPM,

template-matching combined inter-intra prediction (CIIP),

adaptive re-ordering of merge candidates,

temporal motion vector prediction (TMVP) and non-adjacent merge candidate type reordering,

template-matching overlapped block motion compensation (TM-OBMC),

intra template matching prediction (IntraTMP),

intra-block copy-template matching-advanced motion vector prediction (IBC-TM-AMVP), or

intra-block copy-template matching-merge (IBC-TM-AMVP).

11. The device of claim 9 , wherein:

the template-matching tool is a first template-matching tool,

the at least one syntax element is a first syntax element, and

the one or more processors are further configured to signal, based on the first syntax element, a second syntax element that indicates whether a second template-matching tool is enabled.

12. The device of claim 11 , wherein the first template-matching tool is template-matching merge mode and the second template-matching tool is template-matching combined intra/inter prediction (CIIP) mode.

13. The device of claim 11 , wherein:

the first template-matching tool is adaptive reordering of merge candidates (ARMC), and

the second template-matching tool is one of: geometric partitioning mode (GPM) split mode reordering, candidate reordering for regular merge mode with motion vector difference (MMVD) and affine MMVD, motion vector difference (MVD) sign prediction, reference picture reordering, temporal motion vector prediction (TMVP) or non-adjacent merge candidate type reordering.

14. The device of claim 9 , wherein:

the at least one syntax element is a first syntax element, and

the one or more processors are further configured to signal a second syntax element in the bitstream,

the second syntax element indicates whether the bitstream includes one or more template-matching tool syntax elements,

the second syntax element is a sequence-level syntax element, a picture-level syntax element, a slice-level syntax element, or a tile-level syntax element, and

the one or more template-matching tool syntax elements include the first syntax element.

15. The device of claim 14 , wherein presence of syntax elements indicating whether intra template matching prediction (intraTMP), intra block copy template matching with advanced motion vector prediction (IBC-TM-AMVP), and intra block copy template matching with merge mode (IBC-TM-AMVP) are enabled is not dependent on the second syntax element.

16. The device of claim 9 , wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

17. A method of decoding video data, the method comprising:

obtaining at least one syntax element from a bitstream that includes an encoded representation of the video data;

determining, based on the at least one syntax element, that a template-matching tool is enabled;

based on the at least one syntax element indicating intra-block copy template matching merge mode (IBC-TM-MRG) or intra-block copy template matching advance motion vector prediction (IBC-TM-AMVP) is disabled, setting a pruning threshold to a value;

pruning a first candidate from a list based on a difference between a motion vector component of the first candidate and a motion vector component of a second candidate in the list being less than the pruning threshold;

based on the template-matching tool being enabled, applying the template-matching tool to generate a prediction block for a current coding unit (CU) of the video data based on a candidate in the list; and

reconstructing the current CU based on the prediction block for the current CU.

18. The method of claim 17 , wherein the template-matching tool is one of:

template-matching advanced motion vector prediction (TM-AMVP),

geometric partitioning mode (GPM) split mode reordering,

candidate reordering for regular merge mode with motion vector difference (MMVD) and affine MMVD,

motion vector difference (MVD) sign prediction,

reference picture reordering,

template-matching merge mode,

template-matching GPM,

template-matching combined inter-intra prediction (CIIP),

adaptive re-ordering of merge candidates,

temporal motion vector prediction (TMVP) and non-adjacent merge candidate type reordering,

template-matching overlapped block motion compensation (TM-OBMC),

intra template matching prediction (IntraTMP),

intra-block copy-template matching-advanced motion vector prediction (IBC-TM-AMVP), or

intra-block copy-template matching-merge (IBC-TM-AMVP).

19. The method of claim 17 , wherein:

the template-matching tool is a first template-matching tool,

the at least one syntax element is a first syntax element, and

the method further comprises determining, based on the first syntax element, whether the bitstream includes a syntax element that indicates whether a second template-matching tool is enabled.

20. The method of claim 17 wherein:

the at least one syntax element is a first syntax element,

the method further comprises:

obtaining a second syntax element from the bitstream, wherein the second syntax element is a sequence-level syntax element, a picture-level syntax element, a slice-level syntax element, or a tile-level syntax element; and

determining, based on the second syntax element, whether the bitstream includes one or more template-matching tool syntax elements indicating whether one or more template-matching tools are enabled, wherein the one or more template-matching tool syntax elements include the first syntax element.

21. The method of claim 20 , wherein presence of syntax elements indicating whether intra template matching prediction (intraTMP), intra block copy template matching with advanced motion vector prediction (IBC-TM-AMVP), and intra block copy template matching with merge mode (IBC-TM-AMVP) is not dependent on the second syntax element.

22. A method of encoding video data, the method comprising:

based on intra-block copy template matching merge mode (IBC-TM-MRG) or intra-block copy template matching advance motion vector prediction (IBC-TM-AMVP) being disabled, setting a pruning threshold to a value;

pruning a first candidate from a list based on a difference between a motion vector component of the first candidate and a motion vector component of a second candidate in the list being less than the pruning threshold;

based on a template-matching tool being enabled, applying the template-matching tool to generate a prediction block for a current coding unit (CU) of the video data based on a candidate in the list;

encoding the current CU based on the prediction block for the current CU;

signaling, in a bitstream that includes an encoded representation of the video data, at least one syntax element that indicates that the template-matching tool is enabled.

23. The method of claim 22 , wherein the template-matching tool is one of:

template-matching advanced motion vector prediction (TM-AMVP),

geometric partitioning mode (GPM) split mode reordering,

candidate reordering for regular merge mode with motion vector difference (MMVD) and affine MMVD,

motion vector difference (MVD) sign prediction,

reference picture reordering,

template-matching merge mode,

template-matching GPM,

template-matching combined inter-intra prediction (CIIP),

adaptive re-ordering of merge candidates,

temporal motion vector prediction (TMVP) and non-adjacent merge candidate type reordering,

template-matching overlapped block motion compensation (TM-OBMC),

intra template matching prediction (IntraTMP),

intra-block copy-template matching-advanced motion vector prediction (IBC-TM-AMVP), or

intra-block copy-template matching-merge (IBC-TM-AMVP).

24. The method of claim 22 , wherein:

the template-matching tool is a first template-matching tool,

the at least one syntax element is a first syntax element, and

the method further comprises signaling, based on the first syntax element, a second syntax element that indicates whether a second template-matching tool is enabled.

25. The method of claim 24 , wherein:

the first template-matching tool is adaptive reordering of merge candidates (ARMC), and

the second template-matching tool is one of: geometric partitioning mode (GPM) split mode reordering, candidate reordering for regular merge mode with motion vector difference (MMVD) and affine MMVD, motion vector difference (MVD) sign prediction, reference picture reordering, temporal motion vector prediction (TMVP) or non-adjacent merge candidate type reordering.

26. The method of claim 22 , wherein:

the at least one syntax element is a first syntax element,

the method further comprises signaling a second syntax element in the bitstream,

the second syntax element indicates whether the bitstream includes one or more template-matching tool syntax elements,

the second syntax element is a sequence-level syntax element, a picture-level syntax element, a slice-level syntax element, or a tile-level syntax element, and

the one or more template-matching tool syntax elements include the first syntax element.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 7, 2023
From: CHEN, CHUN-CHI; HUANG, HAN; SEREGIN, VADIM; KARCZEWICZ, MARTA
To: QUALCOMM INCORPORATED
Reel/Frame 064512/0467 →
Continuity (2)
Provisional Application 63367793 · Jul 6, 2022
Related Publication 20240015333A1 · Jan 11, 2024
References Cited (21)
US 20190007699A1 · Liu et al. · 2019 [cited by applicant]
US 20230090700A1 · Teng · 2023 [cited by examiner]
WO 2022063729A1 · 2022 [cited by applicant]
Chen C.C., et al., “AHG6: ECM Software Configuration Parameters for Template Matching Tools”, JVET-AA0132-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 27th Meeting, by Teleconference,… [cited by applicant]
Coban M., et al., (Editors): “Algorithm Description of Enhanced Compression Model 5 (ECM 5)”, 138. MPEG Meeting, Apr. 25, 2022-Apr. 29, 2022, Online, (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11), No. M59895, … [cited by applicant]
International Search Report and Written Opinion—PCT/US2023/026077—ISA/EPO—Oct. 9, 2023. [cited by applicant]
Jang H., et al., “Non-EE2: SPS Flag to Control TM-Based Merge/amvp and Multi-pass DMVR Separately”, Teleconference, Apr. 20, 2022-Apr. 29, 2022, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.1… [cited by applicant]
Jang H., et al., “Non-EE2: Support MMVD and Affine MMVD Without Template Matching Process”, Teleconference, Apr. 20, 2022-Apr. 29, 2022, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16 ), No.… [cited by applicant]
Naser K., et al., “Evaluation of Template Matching Prediction for VVC”, Teleconference, Jan. 6, 2021-Jan. 15, 2021, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG. 16 ), No. JVET-U0048, Dec. 28… [cited by applicant]
Ohm J-R: “Meeting Report of the 27th Meeting of the Joint Video Experts Team (JVET)”, Teleconference, Jul. 13, 2022-Jul. 22, 2022, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16), No. JVET-A… [cited by applicant]
Bross B., et al., “Versatile Video Coding Editorial Refinements on Draft 10”, JVET-T2001-v2, 20th JVET Meeting, Oct. 7, 2020-Oct. 16, 2020, Teleconference, (The Joint Video Experts Team of ITU-T SG 16 WP 3 and ISO/IEC J… [cited by applicant]
Browne A., et al., “Algorithm description for Versatile Video Coding and Test Model 17(VTM17)”, JVET-Z2002-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 26th Meeting, by teleconference… [cited by applicant]
Chen C-C., et al., “AHG6: ECM Software Configuration Parameters for Template Matching Tools”, Qualcomm, JVET-AA0132-v1, JVET-AA0132, Jul. 13-22, 2022, 6 Pages. [cited by applicant]
Chen J., et al., “Algorithm Description for Versatile Video Coding and Test Model 11 (VTM 11)”, JVET-T2002-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 20th Meeting, by teleconference… [cited by applicant]
Chen Y-W., et al., “Description of SDR, HDR and 360° Video Coding Technology Proposal by Qualcomm and Technicolor—Low and High Complexity Versions”, JVET-J0021, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 an… [cited by applicant]
Coban M., et al., “Algorithm Description of Enhanced Compression Model 5 (ECM 5)”, JVET-Z2025, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 26th Meeting, by Teleconference, Apr. 20-29, 20… [cited by applicant]
ITU-T H.265: “Series H: Audiovisual and Multimedia Systems Infrastructure of Audiovisual Services—Coding of Moving Video”, High Efficiency Video Coding, The International Telecommunication Union, Jun. 2019, 696 Pages. [cited by applicant]
ITU-T H.266: “Series H: Audiovisual and Multimedia Systems Infrastructure of Audiovisual Services—Coding of Moving Video”, Versatile Video Coding, The International Telecommunication Union, Aug. 2020, 516 pages. [cited by applicant]
Seregin V., et al., “Exploration Experiment on Enhanced Compression beyond VVC capability (EE2)”, JVET-Z2024-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 26th Meeting, by teleconferen… [cited by applicant]
Sullivan G.J., et al., “Overview of the High Efficiency Video Coding (HEVC) Standard”, IEEE Transactions on Circuits and Systems for Video Technology, IEEE Service Center, Piscataway, NJ, US, vol. 22, No. 12, Dec. 1, 20… [cited by applicant]
Wang Y-K., et al., “High Efficiency Video Coding (HEVC) Defect Report”, 14. JCT-VC Meeting; Jul. 25, 2013-Feb. 8, 2013; Vienna; (Joint Collaborative Team on Video Coding of ISO/IEC JTC1/SC29/WG11 and ITU-TSG.16); Url: h… [cited by applicant]
Cited By (2)
US 12,666,047 US 12,744,943