IP Library Granted Patent US 12,368,889
Granted Patent B2
US 12,368,889 · App. 18/312,077 · Granted Jul 22, 2025

Coefficient-based transform and mode signaling

Inventors: Alican Nalci (Cupertino, CA); Yunfei Zheng (Santa Clara, CA); Hilmi E. Egilmez (Santa Clara, CA); Yeqing Wu (Cupertino, CA); Yixin Du (Cupertino, CA); Alexandros Tourapis (Los Gatos, CA); Jun Xin (San Jose, CA); Hsi-Jung Wu (San Jose, CA)
Assignee: APPLE INC.
H04N19/60H04N19/124H04N19/172H04N19/18
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,368,889
App. No.
18/312,077
Granted
Jul 22, 2025
Kind
B2
Abstract

Techniques are described for express and implied signaling of transform mode selections in video coding. Information derived from coefficient samples in a given transform unit (TU) or prediction unit (PU) may constrain or modify signaling of certain syntax elements at the coding block (CB), TU, or PU levels. For instance, based on the spatial locations of decoded coefficients, the spatial patterns of coefficients, or the correlation with the coefficients in neighboring blocks, various syntax elements such as the transform type and related flags/indices or secondary transform modes/flags indices, a residual coding mode, intra and inter prediction modes, and scanning order may be disabled or constrained. In another case, if the coefficient samples match a desired spatial pattern or have other desired properties then a default transform type, a default secondary transform type, a default intra and inter prediction mode or other block level modes may be inferred at the decoder side. Similar ideas may be extended to infer intra-block copy (IBC) flags/modes/indices from coefficient samples, or to infer flags and modes related to predictive coding techniques such as block-wise differential pulse code modulation (BDPCM), i.e., horizontal and vertical BDPCM modes, palette mode syntax and other block level flags/indices.

Claims (38)

1. A video decoding method comprising:

receiving coded data of a transform unit from a channel according to a coding protocol in which an identifier of a position of a last significant quantized transform coefficient of the transform unit precedes an identification of transform type in signaling order;

comparing the position of the last significant quantized transform to a threshold;

assigning a transform type to the transform unit based on the comparison, wherein the assigned transform type is inferred from i) multiple reference line data received from the channel or ii) angle delta offset data received from the channel;

dequantizing received transform coefficients received from the channel; and

transforming the dequantized transform coefficients according to a default transform type.

2. The method of claim 1 , wherein the position is identified by its scan position within the transform unit.

3. The method of claim 1 , wherein the position is identified by its x, y coordinate within the transform unit.

4. The method of claim 1 , wherein when the comparison indicates an implied signaling condition is satisfied, the assigned transform type is the default transform type determined by a governing coding protocol.

5. The method of claim 1 , wherein the assigned transform type is identified in a coding unit level of the coding protocol.

6. The method of claim 1 , wherein the assigned transform type is inferred from a prediction mode of the transform unit.

7. The method of claim 1 , further comprising:

determining if a transform type identifier is present in the received data representing the coded transform unit,

if the transform type identifier is not present in the received data of the coded transform unit, determining one of multiple position ranges to which the position identifier belongs, and determining the coded transform unit's transform type from the determined range.

8. The method of claim 1 , wherein the comparison determines that the position is less than the threshold and, if so, assigns the transform type as a two-dimensional transform.

9. The method of claim 1 , wherein the comparison determines that the position is less than the threshold and, if so, assigns the transform type from a two-member set binary symbol received from a channel.

10. The method of claim 1 , wherein the comparison determines that the position is greater than the threshold and, if so, assigns the transform type as an identity transform.

11. The method of claim 1 , wherein the comparison

determines whether the position is between first and second thresholds and,

if so, compares a transform type received in the channel to an array of candidate transform types, the array being smaller than an entirety of transform types supported by a governing coding protocol, and

selects one of the candidate transform types based on the transform type comparison.

12. The video decoding method of claim 1 , wherein the coding protocol supports coding of transform units in a quadtree representation, and assigned transform type is identified in a transform unit level at a higher level of the quadtree representation than the transform unit level in which the position identifier is received.

13. The method of claim 1 , further comprising deriving a secondary transform type from a processing context of the coded transform unit.

14. A video decoder, comprising:

a processing device,

a memory storing program instructions that, when executed by the processing device, cause the processing device to decode coded video by:

receiving coded data of a transform unit from a channel according to a coding protocol in which an identifier of a position of a last significant quantized transform coefficient of the transform unit precedes an identification of transform type in signaling order;

comparing the position of the last significant quantized transform to a threshold;

assigning a transform type to the transform unit based on the comparison, wherein the assigned transform type is inferred from i) multiple reference line data received from the channel or ii) angle delta offset data received from the channel;

dequantizing received transform coefficients received from the channel;

transforming the dequantized transform coefficients according to the default transform type;

determining if a transform type identifier is present in the received data representing the coded transform unit; and

if the transform type identifier is not present in the received data of the coded transform unit, determining one of multiple position ranges to which the position identifier belongs, and determining the coded transform unit's transform type from the determined range.

15. The decoder of claim 14 , wherein the position is identified by its scan position within the transform unit.

16. The decoder of claim 14 , wherein the position is identified by its x, y coordinate within the transform unit.

17. The decoder of claim 14 , wherein when the comparison indicates an implied signaling condition is satisfied, the assigned transform type is the default transform type determined by a governing coding protocol.

18. The decoder of claim 14 , wherein the assigned transform type is identified in a coding unit level of the coding protocol.

19. The decoder of claim 14 , wherein the assigned transform type is inferred from a prediction mode of the transform unit.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 4, 2023
From: NALCI, ALICAN; ZHENG, YUNFEI; EGILMEZ, HILMI E.; WU, YEQING; DU, YIXIN; TOURAPIS, ALEXIS; XIN, JUN; WU, HSI-JUNG
To: APPLE INC.
Reel/Frame 063538/0979 →
Continuity (2)
Provisional Application 63392943 · Jul 28, 2022
Related Publication 20240040151A1 · Feb 1, 2024
References Cited (40)
US 9191670B2 · Karczewicz · 2015 [cited by examiner]
US 9565404B2 · Su · 2017 [cited by examiner]
US 10638147B2 · Hu · 2020 [cited by examiner]
US 10757428B2 · Guo · 2020 [cited by examiner]
US 10798390B2 · Filippov · 2020 [cited by examiner]
US 10917637B2 · Yoo · 2021 [cited by examiner]
US 11184636B2 · Aono · 2021 [cited by examiner]
US 11582455B2 · Filippov · 2023 [cited by examiner]
US 11632546B2 · Bang · 2023 [cited by examiner]
US 11677950B2 · Schwarz · 2023 [cited by examiner]
US 11882284B2 · Siekmann · 2024 [cited by examiner]
US 11956469B2 · Chiang · 2024 [cited by examiner]
US 11973983B2 · Karczewicz · 2024 [cited by examiner]
US 11991393B2 · Chiang · 2024 [cited by examiner]
US 12126794B2 · Peringassery Krishnan · 2024 [cited by examiner]
US 20140003530A1 · Sole Rojals · 2014 [cited by examiner]
US 20160234499A1 · Chuang · 2016 [cited by examiner]
US 20170064298A1 · He · 2017 [cited by examiner]
US 20170094274A1 · Chien · 2017 [cited by examiner]
US 20180352225A1 · Guo · 2018 [cited by examiner]
US 20210084301A1 · Siekmann · 2021 [cited by examiner]
US 20210112279A1 · Karczewicz · 2021 [cited by examiner]
US 20220046253A1 · Zhang · 2022 [cited by examiner]
US 20220248031A1 · Ma · 2022 [cited by examiner]
US 20230085680A1 · Chujoh · 2023 [cited by examiner]
EP 1478190 · 2004 [cited by examiner]
GB 2503875 · 2012 [cited by examiner]
WO WO2014082088A1 · 2014 [cited by examiner]
WO WO2019083583A1 · 2019 [cited by applicant]
WO WO2019152570A1 · 2019 [cited by applicant]
WO WO2022072062A1 · 2022 [cited by applicant]
ITU-T; Versatile Video Coding (Year: 2022). [cited by examiner]
International Patent Application No. PCT/US2023/070989; Int'l Search Report and the Written Opinion; dated Jan. 8, 2024; 19 pages. [cited by applicant]
U.S. Appl. No. 18/076,166, filed Dec. 6, 2022. [cited by applicant]
Chen et al.; “Algorithm description for Versatile Video Coding and Test Model 12 (VTM 12)”; Document JVET-U2002-v1; Jan. 2021; 101 pages. [cited by applicant]
Lee et al.; “Improved Lossless Intra Coding for H.264/MPEG-4 AVC”; IEEE Transactions on Image Processing; vol. 15 No. 9; Sep. 2006; p. 2610-2615. [cited by applicant]
Nalci et al.; “Forward Skip Coding for Prediction Residuals”; Alliance for Open Media, Codec Working Group; Document CWG-C002-v2; Jan. 2022; 16 pages. [cited by applicant]
Schwarz et al.; “Improved Quantization and Transform Coefficient Coding for the Emerging Versatile Video Coding (VVC) Standard”; IEEE Int'l Conf. on Image Processing; 2019; p. 1183-1187. [cited by applicant]
Browne et al.; “Algorithm description for Versatile Video Coding and Test Model 16 (VTM 16)”; Document JVET-Y2002-v1; Jan. 2022; 126 pages. [cited by applicant]
International Patent Application No. PCT/US2023/070989; Int'l Preliminary Report on Patentability; dated Feb. 6, 2025; 14 pages. [cited by applicant]