IP Library › Granted Patent US 12,707,053
Granted Patent B2
US 12,707,053 · App. 19/240,685 · Granted Aug 11, 2026

Multi-level significance maps for encoding and decoding

Inventors: Nguyen Nguyen (Seattle, WA); Tianying Ji (Albany, CA); Dake He (Waterloo, CA)
H04N19/122H04N19/132H04N19/136H04N19/157H04N19/18H04N19/196H04N19/61H04N19/70H04N19/91H04N19/13H04N19/159H04N19/172
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,707,053
App. No.
19/240,685
Filed
Jun 17, 2025
Granted
Aug 11, 2026
Kind
B2
Art Unit
2482
USPC
375/240.02
Abstract

Methods of encoding and decoding for video data are described in which multi-level significance maps are used in the encoding and decoding processes. The significant-coefficient flags that form the significance map are grouped into contiguous groups, and a significant-coefficient-group flag signifies for each group whether that group contains no non-zero significant-coefficient flags. If there are no non-zero significant-coefficient flags in the group, then the significant-coefficient-group flag is set to zero. The set of significant-coefficient-group flags is encoded in the bitstream. Any significant-coefficient flags that fall within a group that has a significant-coefficient-group flag that is non-zero are encoded in the bitstream, whereas significant-coefficient flags that fall within a group that has a significant-coefficient-group flag that is zero are not encoded in the bitstream.

Claims (78)

1 . An encoding apparatus comprising at least one processor, the at least one processor configured to:

determine, for a first block within a transform unit, that the first block is not a block that contains a position of a DC coefficient in the transform unit;

in response to the determination that the first block is not the block that contains the position of the DC coefficient in the transform unit:

encode, into a bitstream, a significant-coefficient group flag (significant-CG flag) indicating that the first block has at least one non-zero coefficient, by determining a context for the significant-CG flag to be either a first context or a second context, based on values of significant-CG flags corresponding to two blocks which are, respectively, adjacent to the right and below the first block, and wherein the context is determined to be the first context when the values of significant-CG flags of the two blocks are both zero and the context is determined to be the second context otherwise;

encode, for each position in the first block other than a (0,0) position in the first block, a significant-coefficient flag into the bitstream, the significant-coefficient flag indicating whether a coefficient at the respective position in the first block is non-zero;

determine whether any coefficient not at the (0,0) position is non-zero;

in response to a determination that at least one coefficient not at the (0,0) position is non-zero, encode, into the bitstream, a significant-coefficient flag indicating whether a coefficient at the (0,0) position is non-zero; and

in response to a determination that all coefficients not at the (0,0) position are zero, not encode a significant-coefficient flag for the (0,0) position; and

determine, for a second block within the transform unit, that the second block is the block that contains the position of the DC coefficient in the transform unit,

in response to the determination that the second block is the block that contains the position of the DC coefficient in the transform unit:

not encode a significant-CG flag; and

encode, for each position in the second block, a significant-coefficient flag into the bitstream, the significant-coefficient flag indicating whether a coefficient at the respective position in the second block is non-zero.

2 . The encoding apparatus of claim 1 , the at least one processor further configured to:

for the first block within the transform unit:

determine whether any coefficient not at the (0,0) position is non-zero by checking the respective significant-coefficient flag.

3 . The encoding apparatus of claim 1 , the at least one processor further configured to:

for a third block within the transform unit, that contains a position of a last non-zero coefficient in the transform unit:

not encode a significant-CG flag;

encode, for at least one position in the third block that is not the position of the last non-zero coefficient in the transform unit, a significant-coefficient flag into the bitstream, the significant-coefficient flag indicating whether a coefficient at the respective position in the third block is non-zero; and

not encode a significant-coefficient flag for the position of the last non-zero coefficient in the transform unit.

4 . A video encoding method comprising:

determining, for a first block within a transform unit, that the first block is not a block that contains a position of a DC coefficient in the transform unit;

in response to the determination that the first block is not the block that contains the position of the DC coefficient in the transform unit:

encoding, into a bitstream, a significant-coefficient group flag (significant-CG flag) indicating that the first block has at least one non-zero coefficient by determining a context for the significant-CG flag to be either a first context or a second context, based on values of significant-CG flags corresponding to two blocks which are, respectively, adjacent to the right and below the first block, and wherein the context is determined to be the first context when the values of significant-CG flags of the two blocks are both zero and the context is determined to be the second context otherwise;

encoding, for each position in the first block other than a (0,0) position in the first block, a significant-coefficient flag into the bitstream, the significant-coefficient flag indicating whether a coefficient at the respective position in the first block is non-zero;

determining whether any coefficient not at the (0,0) position is non-zero;

in response to a determination that at least one coefficient not at the (0,0) position is non-zero, encoding, into the bitstream, a significant-coefficient flag indicating whether a coefficient at the (0,0) position is non-zero; and

in response to a determination that all coefficients not at the (0,0) position are zero, not encoding a significant-coefficient flag for the (0,0) position; and

determining, for a second block within the transform unit, that the second block is the block that contains the position of the DC coefficient in the transform unit;

in response to the determination that the second block is the block that contains the position of the DC coefficient in the transform unit:

not encoding a significant-CG flag; and

encoding, for each position in the second block, a significant-coefficient flag into the bitstream, the significant-coefficient flag indicating whether a coefficient at the respective position in the second block is non-zero.

5 . The video encoding method of claim 4 , further comprising:

for the first block within the transform unit:

determining whether any coefficient not at the (0,0) position is non-zero by checking the respective significant-coefficient flag.

6 . The video encoding method of claim 4 , further comprising:

for a third block within the transform unit, that contains a position of a last non-zero coefficient in the transform unit:

not encoding a significant-CG flag;

encoding, for at least one position in the third block that is not the position of the last non-zero coefficient in the transform unit, a significant-coefficient flag into the bitstream, the significant-coefficient flag indicating whether a coefficient at the respective position in the third block is non-zero; and

not encoding a significant-coefficient flag for the position of the last non-zero coefficient in the transform unit.

7 . A decoding apparatus comprising at least one processor, the at least one processor configured to:

determine, for a first block within a transform unit, that the first block is not a block that contains a position of a DC coefficient in the transform unit;

in response to the determination that the first block is not the block that contains the position of the DC coefficient in the transform unit:

decode a significant-coefficient group flag (significant-CG flag) indicating that the first block has at least one non-zero coefficient by determining a context for the significant-CG flag to be either a first context or a second context, based on values of significant-CG flags corresponding to two blocks which are, respectively, adjacent to the right and below the first block, and wherein the context is determined to be the first context when the values of significant-CG flags of the two blocks are both zero and the context is determined to be the second context otherwise;

decode, for each position in the first block other than a (0,0) position in the first block, a significant-coefficient flag indicating whether a coefficient at the respective position in the first block is non-zero;

determine whether any coefficient not at the (0,0) position is non-zero;

in response to a determination that at least one coefficient not at the (0,0) position is non-zero, decode a significant-coefficient flag indicating whether a coefficient at the (0,0) position is non-zero; and

in response to a determination that all coefficients not at the (0,0) position are zero, not decode a significant-coefficient flag for the (0,0) position and infer that a coefficient at the (0,0) position is non-zero; and

determine for a second block within the transform unit, that the second block is the block that contains the position of the DC coefficient in the transform unit;

in response to the determination that the second block is the block that contains the position of the DC coefficient in the transform unit:

not decode a significant-CG flag; and

decode, for each position in the second block, a significant-coefficient flag indicating whether a coefficient at the respective position in the second block is non-zero.

8 . The decoding apparatus of claim 7 , the at least one processor further configured to: for the first block within the transform unit:

determine whether any coefficient not at the (0,0) position is non-zero by checking the respective significant-coefficient flag.

9 . The decoding apparatus of claim 7 , the at least one processor further configured to: for a third block within the transform unit, that contains a position of a last non-zero coefficient in the transform unit:

not decode a significant-CG flag;

decode, for at least one position in the third block that is not the position of the last non-zero coefficient in the transform unit, a significant-coefficient flag, the significant-coefficient flag indicating whether a coefficient at the respective position in the third block is non-zero; and

not decode a significant-coefficient flag for the position of the last non-zero coefficient in the transform unit.

10 . A video decoding method comprising:

determining, for a first block within a transform unit, that the first block is not a block that contains a position of a DC coefficient in the transform unit;

in response to the determination that the first block is not the block that contains the position of the DC coefficient in the transform unit:

decoding a significant-coefficient group flag (significant-CG flag) indicating that the first block has at least one non-zero coefficient by determining a context for the significant-CG flag to be either a first context or a second context, based on values of significant-CG flags corresponding to two blocks which are, respectively, adjacent to the right and below the first block, and wherein the context is determined to be the first context when the values of significant-CG flags of the two blocks are both zero and the context is determined to be the second context otherwise;

decoding, for each position in the first block other than a (0,0) position in the first block, a significant-coefficient flag indicating whether a coefficient at the respective position in the first block is non-zero;

determining whether any coefficient not at the (0,0) position is non-zero;

in response to a determination that at least one coefficient not at the (0,0) position is non-zero, decoding a significant-coefficient flag indicating whether a coefficient at the (0,0) position is non-zero; and

in response to a determination that all coefficients not at the (0,0) position are zero, not decoding a significant-coefficient flag for the (0,0) position, and inferring that a coefficient at the (0,0) position is non-zero; and

determining, for a second block within the transform unit, that the second block is the block that contains the position of the DC coefficient in the transform unit;

in response to the determination that the second block is the block that contains the position of the DC coefficient in the transform unit:

not decoding a significant-CG flag; and

decoding, for each position in the second block, a significant-coefficient flag indicating whether a coefficient at the respective position in the second block is non-zero.

11 . The decoding method of claim 10 , further comprising:

for the first block within the transform unit:

determining whether any coefficient not at the (0,0) position is non-zero by checking the respective significant-coefficient flag.

12 . The decoding method of claim 10 , further comprising:

for a third block within the transform unit, that contains a position of a last non-zero coefficient in the transform unit:

not decoding a significant-CG flag;

decoding, for at least one position in the third block that is not the position of the last non-zero coefficient in the transform unit, a significant-coefficient flag, the significant-coefficient flag indicating whether a coefficient at the respective position in the third block is non-zero; and

not decoding a significant-coefficient flag for the position of the last non-zero coefficient in the transform unit.

Continuity (9)
Continuation 19216590 · May 22, 2025
Continuation 17888377 · Aug 15, 2022
Continuation 17164766 · Feb 1, 2021
Continuation 16865272 · May 1, 2020
Continuation 16226870 · Dec 20, 2018
Continuation 15884535 · Jan 31, 2018
Continuation 14621552 · Feb 13, 2015
Continuation 13286336 · Nov 1, 2011
Related Publication 20250373802A1 · Dec 4, 2025
References Cited (55)
US 8964849B2 · Nguyen · 2015 [cited by applicant]
US 10116950B2 · He · 2018 [cited by applicant]
US 10659782B2 · Nguyen · 2020 [cited by applicant]
US 10681362B2 · He · 2020 [cited by applicant]
US 20070041450A1 · Kim · 2007 [cited by applicant]
US 20070071331A1 · Liu · 2007 [cited by applicant]
US 20070110153A1 · Cho · 2007 [cited by applicant]
US 20070160133A1 · Bao · 2007 [cited by applicant]
US 20080219578A1 · Lee · 2008 [cited by applicant]
US 20090175332A1 · Karczewicz · 2009 [cited by applicant]
US 20100086030A1 · Chen et al. · 2010 [cited by applicant]
US 20110009683A1 · Lebedev · 2011 [cited by applicant]
US 20110096834A1 · Cheon · 2011 [cited by examiner]
US 20110200110A1 · Chen · 2011 [cited by applicant]
US 20110249721A1 · Karczewicz · 2011 [cited by applicant]
US 20120082233A1 · Sze · 2012 [cited by applicant]
US 20130016771A1 · Misra · 2013 [cited by applicant]
US 20130051475A1 · Joshi · 2013 [cited by applicant]
US 20130083855A1 · Kottke · 2013 [cited by applicant]
EP 2154894A1 · 2010 [cited by applicant]
JP 2004064725A · 2004 [cited by applicant]
JP 02239776A · 2007 [cited by applicant]
WO 2008108534A1 · 2008 [cited by applicant]
WO 2010018138A1 · 2010 [cited by applicant]
WO 2011128303A2 · 2011 [cited by applicant]
Australian Patent Office, AU Office Action relating to Application No. 2012244144, dated Oct. 30, 2013. [cited by applicant]
B. Bross, W-J Han, J-R Ohm, G. J. Sullivan, and T. Wiegand, “WD4: Working Draft 4 of High-Efficiency Video Coding,” JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 6th Meeting, Torino, Jul. 2011. [cited by applicant]
Blackberry Limited''s and Blackberry Corporation''s Complaint for Patent Infringement, Demand for Jury Trial, filed Jul. 27, 2016. [cited by applicant]
Bross, B. et al., “WD4: Working Draft 4 of High-Efficiency Video Coding”, 6 JCT-VC Meeting; 97 MPEG Meeting; Jul. 14, 2011-Jul. 22, 2011; Torino; (Joint Collaborative Team on Video Coding of ISO/IEC JTC1/SC29/WG11 and I… [cited by applicant]
EPO Partial Search Report relating to application No. 11187410.3, dated Feb. 15, 2012. [cited by applicant]
EPO, EP Office Action relating to EP application No. 14200316.9, dated May 30, 2018. [cited by applicant]
EPO, European Office Action dated May 2, 2013. [cited by applicant]
EPO, Extended European Search Report relating to application No. 11187410.3 dated Jun. 19, 2012. [cited by applicant]
EPO, Extended European Search Report relating to application No. 12164134.4, dated Feb. 15, 2013. [cited by applicant]
EPO, Extended European Search Report relating to Application No. 14200316.9, dated Apr. 16, 2015. [cited by applicant]
European Search Report received in European Application No. 19152375.2, dated May 16, 2019, 14 pages. [cited by applicant]
F. Bossen, “Common test conditions and software reference configurations”, JCTVC-F900, JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 6th Meeting, Torino, Jul. 2011. [cited by applicant]
Iain E. Richardson, “The H.264 Advanced Video Compression Standard, 2nd Edition, Chapter 3, Video Coding Concepts”, Apr. 20, 2010, XP2126854A. [cited by applicant]
Intellectual Property India, Examination Report relating to application No. 4376/CHE/2012, dated Jan. 17, 2018. [cited by applicant]
JPO, Japanese Office Action relating to JP application No. 2012-239852, dated Sep. 30, 2013. [cited by applicant]
KIPO, KR Office Action relating to KR application No. 10-2012-0122364 dated Nov. 1, 2013. [cited by applicant]
Marpe D. et al., “Context-based adaptive binary arithmetic coding in the H.264/AVC video compression standard”, IEEE Transactions on Circuits and Systems for Video Technology, IEEE Service Center, Piscataway, NJ, US, vo… [cited by applicant]
Marpe, H. Schwarz, and T. Wiegand, “Context-based adaptive binary arithmetic coding in the h.264/AVC video compression standard,” IEEE Transactions on Circuits and Systems for Video Technology, 13(7):620-636, Jul. 2003. [cited by applicant]
Nguyen, N. et al., “Multi-level Significant Maps for Large Transform Units”, 7. JCT-VC Meeting; 98. MPEG Meeting; Nov. 21, 2011-Nov. 30, 2011; Geneva; (Joint Collaborative Team on Video Coding of ISO/EIC JTC1/SC29/WG11 … [cited by applicant]
Office Action received in Chinese Application No. 201610335065.0, dated Jan. 31, 2019, 9 pages. [cited by applicant]
Pennebaker, et al., “JPEG Still Image Data Compression Standard”, Jan. 1, 1992, XP2126854A. [cited by applicant]
Richardson, Iain E. G., H264/MPEG-4 Part 10 White Paper—Prediction of Intra Macroblocks, XP002281494, www.vcodex.com, dated Apr. 30, 2003, 6 pages. [cited by applicant]
SIPO, CN Office Action relating to CN application No. 201610335062.0, dated Jun. 1, 2018. [cited by applicant]
SIPO, CN Office Action relating to CN application No. 201610335065.0, dated Jun. 1, 2018. [cited by applicant]
T. Nguyen, T. Winken, D. Marpe, et al., “Reduced-complexity entropy coding of transform coefficient levels using a combination of VLC and Pipe”, JCTVC-D336, JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SG29/WG11, 4th Meeti… [cited by applicant]
T. Nguyen, T. Winken, D. Marpe, et al., “Reduced-Complexity Entropy Coding of Transform Coefficient Levels Using a Combination of VLC and PIPE”, May 9, 2014. [cited by applicant]
United States District Court, Northern District Of Texas, Dallas Division, Case 3:16-cv-02185-G Document 1 Filed Jul. 27, 2016 p. 1 of 115. [cited by applicant]
USPTO letter dated Aug. 18, 2016. [cited by applicant]
W.J. Han, J-R Ohm, G. J. Sullivan, and T. Wiegand, “WD4: Working Draft of High-Efficiency Video Coding,” JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WVG11, 6th Meeting, Torino, Jul. 2011. [cited by applicant]
ITU-T H.264 (May 2003)—Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video. [cited by applicant]