IP Library › Granted Patent US 12,368,887
Granted Patent B2
US 12,368,887 · App. 18/622,771 · Granted Jul 22, 2025

Secondary transform application for various block sizes

Inventors: Xin Zhao (San Jose, CA); Xiang Li (Saratoga, CA); Shan Liu (San Jose, CA)
Assignee: TENCENT AMERICA LLC
H04N19/593H04N19/176H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,368,887
App. No.
18/622,771
Granted
Jul 22, 2025
Kind
B2
Abstract

Aspects of the disclosure provide methods, apparatuses, and non-transitory computer-readable storage mediums for video encoding/decoding. In a method, prediction information for a current block is decoded. The prediction information indicates a first intra prediction mode and a secondary transform index, based on which a secondary transform core is determined. A first transform coefficient block is de-quantized from the prediction information. A size of the first transform coefficient block is less than a size of the secondary transform core. A part of a second transform coefficient block is generated based on the first transform coefficient block and the secondary transform core. A size of the second transform coefficient block equals the size of the secondary transform core. A size of the part of the second transform coefficient equals the size of the first transform coefficient block. The current block is reconstructed based on the part of the second transform coefficient block.

Claims (55)

1. A method for video encoding, comprising:

selecting a secondary transform core for coding a current block in a current picture, the secondary transform core having a size of M×N;

applying a forward primary transform to a transform unit of the current block to generate a primary transform coefficient block having a size of W×H, wherein one of H or W is less than both M and N;

applying the secondary transform core having the size of M×N to the primary transform coefficient block having the size of W×H by applying a sub-section of the secondary transform core to the primary transform coefficient block and generating a secondary transform coefficient block; and

encoding the current block based on an intra prediction mode and the secondary transform coefficient block.

2. The method of claim 1 , wherein the generating the secondary transform coefficient block comprises:

determining a value at a coordinate position of the secondary transform coefficient block based on a value at a same coordinate position of the primary transform coefficient block.

3. The method of claim 1 , further comprising including, in syntax elements of the secondary transform coefficient block, a syntax element that indicates a secondary transform index indicating the secondary transform core.

4. The method of claim 1 , wherein the selecting the secondary transform core comprises:

selecting the secondary transform core based on a mode number of the intra prediction mode, and another intra prediction mode adjacent to the intra prediction mode.

5. The method of claim 1 , further comprising:

determining a context used for entropy coding of a secondary transform index indicating the secondary transform core based on a mode number of the intra prediction mode.

6. An apparatus for video decoding, comprising:

processing circuitry configured to:

decode prediction information for a current block in a current picture that is a part of a coded video sequence, the prediction information indicating an intra prediction mode and a secondary transform index for the current block;

select a secondary transform core based on the intra prediction mode and the secondary transform index, the secondary transform core having a size of M×N;

de-quantize transform coefficients from the prediction information to generate a secondary transform coefficient block;

apply the secondary transform core having the size of M×N to the secondary transform coefficient block by applying a sub-section of the secondary transform core to the secondary transform coefficient block and generate a W×H primary transform coefficient block, wherein one of H or W is less than both M and N; and

reconstruct the current block based on the primary transform coefficient block.

7. The apparatus of claim 6 , wherein the processing circuitry is further configured to:

determine a value at a coordinate position of the W×H primary transform coefficient block based on a value at a same coordinate position of the secondary transform coefficient block.

8. The apparatus of claim 6 , wherein syntax elements of the secondary transform coefficient block include a syntax element that indicates the secondary transform index.

9. The apparatus of claim 6 , wherein the processing circuitry is further configured to:

determine the secondary transform core based on the secondary transform index, a mode number of the intra prediction mode, and another intra prediction mode adjacent to the intra prediction mode.

10. A method of processing visual media data, the method comprising:

processing a bitstream of the visual media data according to a format rule, wherein

the bitstream includes coding information of a current block, the coding information indicating an intra prediction mode and a secondary transform index;

the format rule specifies that a secondary transform core is selected based on the intra prediction mode and the secondary transform index, the secondary transform core having a size of M×N;

the format rule specifies that transform coefficients are de-quantized from the bitstream to generate a secondary transform coefficient block;

the format rule specifies that the secondary transform core having the size of M×N is applied to the secondary transform coefficient block by applying a sub-section of the secondary transform core to the secondary transform coefficient block and generating a W×H primary transform coefficient block, wherein one of H or W is less than both M and N; and

the format rule specifies that the current block of the visual media data is reconstructed based on the primary transform coefficient block.

11. The method of claim 1 , wherein H or W is less than 8.

12. The method of claim 11 , wherein W×H is one of 2×H, W×2, 6×H, and W×6.

13. The method of claim 1 , wherein

W×H is one of 8×4 and 4×8, and

the generating the secondary transform coefficient block includes calculating last L transform coefficients in a coefficient parsing order and setting remaining transform coefficients as 0, L being less than or equal to 8.

14. The method of claim 1 , wherein

W×H is one of L×4 and 4×L, L is larger than 8, and M×N is 8×8, and

the generating the secondary transform coefficient block includes generating 16 nonzero transform coefficients.

15. The method of claim 3 , wherein

the syntax element that indicates the secondary transform index is signaled before transform coefficients in the secondary transform coefficient block,

W and H are less than or equal to 8 and are greater than 2, and

the encoding includes encoding only first 8 transform coefficients of the transform coefficients along a scanning order and not encoding syntax elements of remaining transform coefficients of the transform coefficients.

16. The method of claim 3 , wherein

the syntax element that indicates the secondary transform index is signaled before transform coefficients in the secondary transform coefficient block,

W and H are greater than 4, and

the encoding includes encoding only first 16 transform coefficients of the transform coefficients along a scanning order and not encoding syntax elements of remaining transform coefficients of the transform coefficients.

17. The apparatus of claim 6 , wherein H or W is less than 8.

18. The apparatus of claim 17 , wherein W×H is one of 2×H, W×2, 6×H, and W×6.

19. The apparatus of claim 6 , wherein

W×H is one of 8×4 and 4×8, and

the secondary transform coefficient block includes last L transform coefficients in a coefficient parsing order and remaining transform coefficients are 0, L being less than or equal to 8.

20. The apparatus of claim 6 , wherein

W×H is one of L×4 and 4×L, L is larger than 8, and M×N is 8×8, and

the secondary transform coefficient block includes 16 nonzero transform coefficients and remaining transform coefficients are 0.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 18, 2024
From: ZHAO, XIN; LI, XIANG; LIU, SHAN
To: TENCENT AMERICA LLC
Reel/Frame 067150/0722 →
Continuity (6)
Continuation 17497511 · Oct 8, 2021
Continuation 16889738 · Jun 1, 2020
Provisional Application 62857125 · Jun 4, 2019
Provisional Application 62877727 · Jul 23, 2019
Provisional Application 62897226 · Sep 6, 2019
Related Publication 20240251102A1 · Jul 25, 2024
References Cited (45)
US 9088770B2 · Zhang · 2015 [cited by examiner]
US 10491922B2 · Zhao · 2019 [cited by examiner]
US 11095893B2 · Hsieh · 2021 [cited by examiner]
US 11218728B2 · Zhao et al. · 2022 [cited by applicant]
US 11425421B1 · Koo · 2022 [cited by examiner]
US 20140050266A1 · Zhang · 2014 [cited by examiner]
US 20170280162A1 · Zhao · 2017 [cited by examiner]
US 20170324643A1 · Seregin · 2017 [cited by examiner]
US 20170359595A1 · Zhang et al. · 2017 [cited by applicant]
US 20180103252A1 · Hsieh · 2018 [cited by examiner]
US 20180302631A1 · Chiang · 2018 [cited by examiner]
US 20190149822A1 · Kim · 2019 [cited by examiner]
US 20200177889A1 · Kim et al. · 2020 [cited by applicant]
US 20200366937A1 · Egilmez · 2020 [cited by examiner]
US 20210014534A1 · Koo et al. · 2021 [cited by applicant]
US 20210084301A1 · Siekmann · 2021 [cited by examiner]
US 20220103824A1 · Koo · 2022 [cited by examiner]
US 20220210426A1 · Koo · 2022 [cited by examiner]
US 20220329809A1 · Huo · 2022 [cited by examiner]
EP 3457691A1 · 2019 [cited by examiner]
EP 3764649A1 · 2021 [cited by examiner]
WO WO2019194504A1 · 2019 [cited by examiner]
WO WO2021194221A1 · 2021 [cited by examiner]
Chen et al. Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5), Joint Video Experts Team (JVET) of IYU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, CH, Mar. 19-27, 2019 (Year: … [cited by examiner]
Appendix A, JVET-O0292 Results Table 3 (22 pages). [cited by applicant]
Appendix B, JVET-O0292 Results Table 4 (24 pages). [cited by applicant]
Appendix C, JVET-O0292 Results Table 5 (24 pages). [cited by applicant]
Appendix D, JVET-00350 and O0349-v2 Results (8 pages). [cited by applicant]
Appendix E, JVET-O0350-v2 Results (8 pages). [cited by applicant]
Brass et al.,—“Versatile Video Coding (Draft 5),” JVET-N1001-v9, 14th Meeting: Geneva, CH, Mar. 19-27, 2019 (406 pages). [cited by applicant]
Chen et al. Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5), Joint Video Experts Team (JVET) of IYU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WVG 11 14th Meeting: Geneva, CH, Mar. 19-27, 2019 (Year:… [cited by applicant]
Chen et al., “Algorithm description for Versatile Video Coding and Test Model 5 (VTM-5),” JVET-N1002-v1 [May 21, 2019], Joint Video Experts Team (JVET), 14th Meeting: Geneva, Switzerland, Mar. 19-27, 2019 [retrieved on … [cited by applicant]
Chiang et al., “CE6-related: Latency reduction for LFNST signalling,” JVET-00293-v5 [Jul. 9, 2019], Joint Video Experts Team (JVET) 15th Meeting: Gothenburg, Sweden, Jul. 3-12, 2019 [retrieved Jul. 9, 2019]. Retrieved f… [cited by applicant]
Chiang et al., “CE6-related: Simplifications for LFNST.” JVET-O0292-v1, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019 (6 pages). [cited by applicant]
Chiang et al., “CE6-related: Simplifications for LFNST.” JVET-O0292-v2, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019 (8 pages). [cited by applicant]
Chiang et al., “CE6-related: Simplifications for LFNST.” JVET-O0292-v3, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019 (7 pages). [cited by applicant]
Chiang et al., “JVET-O0292: CE6-related: Simplifications for LFNST,” (7 pages). [cited by applicant]
Extended European Search Report in EP20818014, mailed Apr. 26, 2022, 16 pages. [cited by applicant]
International Search Report and Written Opinion issued Aug. 18, 2020 in International Application No. PCT/US2020/035953, (10 pages). [cited by applicant]
Lainema et al., “CE6-related: LFNST with one mode,” JVET-O0350, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019 (4 pages). [cited by applicant]
Nokia Technologies, “CE6-related: LFNST with one mode JVET-O0350,” (4 pages). [cited by applicant]
Office Action issued in U.S. Appl. No. 17/497,511, mailed Aug. 3, 2023, 19 pages. [cited by applicant]
Partial Supplementary European Search Report in EP20818014, mailed Jan. 19, 2022, 13 pages. [cited by applicant]
Zhang et al., “Non-CE6: On LFNST transform set selection for a CCLM coded block,” JVET-O0219-v1 [Jun. 24, 2019], Joint Video Experts Team (JVET) 15th Meeting: Gothenburg, Sweden, Jul. 3-12, 2019 [retrieved Jun, 24, 2019… [cited by applicant]
Zhao et al., “CE6-related: Unified LFNST using block size independent kernel,” JVET-O0539-v2 [Jul. 6, 2019], Joint Video Experts Team (JVET) 15th Meeting: Gothenburg, Sweden, Jul. 3-12, 2019 [retrieved Jul. 6, 2019]. Re… [cited by applicant]