IP Library Granted Patent US 12,192,514
Granted Patent B2
US 12,192,514 · App. 17/515,237 · Granted Jan 7, 2025

Method and apparatus for video coding

Inventors: Xin Zhao (Santa Clara, CA); Xiang Li (Saratoga, CA); Shan Liu (San Jose, CA)
Assignee: TENCENT AMERICA LLC
H04N19/593H04N19/176H04N19/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,192,514
App. No.
17/515,237
Granted
Jan 7, 2025
Kind
B2
Abstract

Aspects of the disclosure provide methods, apparatuses, and non-transitory computer-readable storage mediums for video encoding/decoding. An apparatus includes processing circuitry that generates a first primary transform coefficient block for a current block based on a primary transform core of the current block. The processing circuitry determines whether a secondary transform is to be applied to the first primary transform coefficient block based on a position of a non-zero transform coefficient in the first primary transform coefficient block. Based on the secondary transform being applied to the first primary transform coefficient block, the processing circuitry generates a secondary transform coefficient block based on the first primary transform coefficient block and a secondary transform core of the current block. A size of the secondary transform core is greater than a size of the first primary transform coefficient block. The processing circuitry encodes the current block based on the secondary transform coefficient block.

Claims (54)

1. A method for video encoding, comprising:

generating a first primary transform coefficient block for a current block based on a primary transform core of the current block;

determining whether to transpose the first primary transform coefficient block based on a type of one-dimensional cross-component linear model applied to the current block;

in response to a determination to transpose the first primary transform coefficient block, transposing the first primary transform coefficient block;

generating a second primary transform coefficient block larger in size than the first primary transform coefficient block by including, in the second primary transform coefficient block, values of the first primary transform coefficient block and a number of 0 values to make a size of the second primary transform coefficient block equal to a size of a secondary transform core;

generating a secondary transform coefficient block based on (i) the second primary transform coefficient block and (ii) the secondary transform core of the current block, the size of the secondary transform core being greater than a size of the first primary transform coefficient block, and

encoding the current block based on the secondary transform coefficient block.

2. The method of claim 1 , wherein the generating the second primary transform coefficient block comprises:

initializing the second primary transform coefficient block with a value at each coordinate position being 0; and

determining a value at a coordinate position of a part of the second primary transform coefficient block based on a value at a same coordinate position of the first primary transform coefficient block, a size of the part of the second primary transform coefficient block being equal to the size of the first primary transform coefficient block.

3. The method of claim 1 , further comprising:

determining whether a position of a non-zero transform coefficient in the first primary transform coefficient block is greater than a predetermined position along a forward coefficient scanning order; and

determining that a secondary transform is to be applied to the first primary transform coefficient block based on the position of the non-zero transform coefficient in the first primary transform coefficient block not being greater than the predetermined position along the forward coefficient scanning order.

4. The method of claim 1 , further comprising:

selecting a secondary transform index to indicate the secondary transform core of the current block; and

encoding the secondary transform index into a video bitstream.

5. The method of claim 4 , wherein the secondary transform index is signaled after a last non-zero transform coefficient of the secondary transform coefficient block and before one or more syntax elements related to coefficient coding of the secondary transform coefficient block.

6. The method of claim 4 , wherein whether a syntax element of the secondary transform coefficient block is signaled is dependent on the secondary transform index and a transform coefficient associated with the syntax element.

7. The method of claim 1 , wherein a syntax element indicating one or more primary transform cores for the current block is signaled after a last non-zero transform coefficient of the secondary transform coefficient block and before one or more syntax elements related to coefficient coding of the secondary transform coefficient block.

8. The method of claim 4 , further comprising:

determining a context used for entropy coding of the secondary transform index based on a shape of the secondary transform core.

9. The method of claim 4 , wherein the determining the secondary transform index comprises:

determining the secondary transform index based on the secondary transform core and a mode number of an intra prediction mode of the current block.

10. The method of claim 4 , further comprising:

determining a context used for entropy coding of the secondary transform index based on a mode number of an intra prediction mode of the current block.

11. An apparatus, comprising:

processing circuitry configured to:

generate a first primary transform coefficient block for a current block based on a primary transform core of the current block;

determine whether to transpose the first primary transform coefficient block based on a type of one-dimensional cross-component linear model applied to the current block;

in response to a determination to transpose the first primary transform coefficient block, transpose the first primary transform coefficient block;

generate a second primary transform coefficient block larger in size than the first primary transform coefficient block by including, in the second primary transform coefficient block, values of the first primary transform coefficient block and a number of 0 values to make a size of the second primary transform coefficient block equal to a size of a secondary transform core;

generate a secondary transform coefficient block based on (i) the second primary transform coefficient block and (ii) the secondary transform core of the current block, the size of the secondary transform core being greater than a size of the first primary transform coefficient block, and

encode the current block based on the secondary transform coefficient block.

12. The apparatus of claim 11 , wherein the processing circuitry is configured to:

initialize the second primary transform coefficient block with a value at each coordinate position being 0; and

determine a value at a coordinate position of a part of the second primary transform coefficient block based on a value at a same coordinate position of the first primary transform coefficient block, a size of the part of the second primary transform coefficient block being equal to the size of the first primary transform coefficient block.

13. The apparatus of claim 11 , wherein the processing circuitry is configured to:

determine whether a position of a non-zero transform coefficient in the first primary transform coefficient block is greater than a predetermined position along a forward coefficient scanning order; and

determine that a secondary transform is to be applied to the first primary transform coefficient block based on the position of the non-zero transform coefficient in the first primary transform coefficient block not being greater than the predetermined position along the forward coefficient scanning order.

14. The apparatus of claim 11 , wherein the processing circuitry is configured to:

select a secondary transform index to indicate the secondary transform core of the current block; and

encode the secondary transform index into a video bitstream.

15. The apparatus of claim 14 , wherein the secondary transform index is signaled after a last non-zero transform coefficient of the secondary transform coefficient block and before one or more syntax elements related to coefficient coding of the secondary transform coefficient block.

16. The apparatus of claim 14 , wherein whether a syntax element of the secondary transform coefficient block is signaled is dependent on the secondary transform index and a transform coefficient associated with the syntax element.

17. The apparatus of claim 11 , wherein a syntax element indicating one or more primary transform cores for the current block is signaled after a last non-zero transform coefficient of the secondary transform coefficient block and before one or more syntax elements related to coefficient coding of the secondary transform coefficient block.

18. A method of processing visual media data, the method comprising:

processing a bitstream of the visual media data according to a format rule, wherein

the bitstream includes coded information of a current block,

the format rule specifies that a first primary transform coefficient block is generated for the current block based on a primary transform core of the current block;

the format rule specifies that whether to transpose the first primary transform coefficient block is determined based on a type of one-dimensional cross-component linear model applied to the current block;

the format rule specifies that, in response to a determination to transpose the first primary transform coefficient block, transposing the first primary transform coefficient block;

generating a second primary transform coefficient block larger in size than the first primary transform coefficient block by including, in the second primary transform coefficient block, values of the first primary transform coefficient block and a number of 0 values to make a size of the second primary transform coefficient block equal to a size of a secondary transform core;

generating a secondary transform coefficient block based on (i) the second primary transform coefficient block and (ii) the secondary transform core of the current block, the size of the secondary transform core being greater than a size of the first primary transform coefficient block, and

encoding the current block based on the secondary transform coefficient block.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 20, 2024
From: ZHAO, XIN; LI, XIANG; LIU, SHAN
To: TENCENT AMERICA LLC
Reel/Frame 068650/0722 →
Continuity (6)
Continuation 17497511 · Oct 8, 2021
Continuation 16889738 · Jun 1, 2020
Provisional Application 62897226 · Sep 6, 2019
Provisional Application 62877727 · Jul 23, 2019
Provisional Application 62857125 · Jun 4, 2019
Related Publication 20220053215A1 · Feb 17, 2022
References Cited (36)
US 9086770B2 · Zhang et al. · 2015 [cited by applicant]
US 10491922B2 · Zhao et al. · 2019 [cited by applicant]
US 11095893B2 · Hsieh · 2021 [cited by examiner]
US 20140050266A1 · Zhang · 2014 [cited by examiner]
US 20170280162A1 · Zhao · 2017 [cited by applicant]
US 20170324643A1 · Seregin · 2017 [cited by examiner]
US 20170359695A1 · Zhang et al. · 2017 [cited by applicant]
US 20180103262A1 · Hsich et al. · 2018 [cited by applicant]
US 20180302631A1 · Chiang et al. · 2018 [cited by applicant]
US 20190149822A1 · Kim et al. · 2019 [cited by applicant]
US 20200366937A1 · Egilmez · 2020 [cited by examiner]
US 20210014534A1 · Koo et al. · 2021 [cited by applicant]
EP 3457691A1 · 2019 [cited by examiner]
EP 3764649A1 · 2021 [cited by examiner]
WO WO2019194504A1 · 2019 [cited by examiner]
Chen et al. Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5), Joint Video Experts Team (JVET) of IYU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 14th Meeting: Geneva, CH, Mar. 19-27, 2019 (Year: … [cited by examiner]
Bross et al., “Versatile Video Coding (Draft 5),” JVET-N1001-v9, 14th Meeting: Geneva, CH, Mar. 19-27, 2019 (406 pages). [cited by applicant]
Chiang et al., “CE6-related: Simplifications for LFNST.” JVET-00292-v1. 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019 (6 pages). [cited by applicant]
Chiang et al., “CE6-related: Simplifications for LFNST.” JVET-00292-v2, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019 (8 pages). [cited by applicant]
Chiang et al., “CES-related: Simplifications for LFNST.” JVET-O0292-3, 15th Meeting; Gothenburg. SE, Jul. 3-12, 2019 (7 pages). [cited by applicant]
Lainema et al., “CE6-related: LFNST with one mode,” JVET-00350, 15th Meeting: Gothenburg. SE, Jul. 3-12, 2019 (4 pages). [cited by applicant]
Appendix A, JVET-O0292 Results Table 3 (22 pages). [cited by applicant]
Appendix B, JVET-O0292 Results Table 4 (24 pages). [cited by applicant]
Appendix C, JVET-O0282 Results Table 5 (24 pages). [cited by applicant]
Chiang et al., “JVET-O0292. CE6-related: Simplifications for LFNST.” (7 pages). [cited by applicant]
Nokia Technologies, “CES-related: LFNST with one mode JVET-O0350,” (4 pages). [cited by applicant]
Appendix D, JVET-O0350 and O0349-v2 Results (8 pages). [cited by applicant]
Appendix E. JVET-O0350-v2 Results (8 pages). [cited by applicant]
International Search Report adn Written Opinion issued Aug. 18, 2020 in international Application No. PCT/US2020/035953, 10 pages. [cited by applicant]
Zhao et al., “CE6-related: Unified LFNST using block size independent kernel,” JVET-O0539-v2 [Jul. 6, 2019], Joint Video Experts Team (JVET) 15th Meeting: Gothenburg, Sweden, Jul. 3-12, 2019 [retrieved Jul. 6, 2019]. Re… [cited by applicant]
Zhang et al., “Non-CE6: On LFNST transform set selection for a CCLM coded block,” JVET-O0219-v1 [Jun. 24, 2019], Joint Video Experts Team (JVET) 15th Meeting: Gothenburg, Sweden, Jul. 3-12, 2019 [retrieved Jun. 24, 2019… [cited by applicant]
Partial Supplementary European Search Report in EP20818014, mailed Jan. 19, 2022, 13 pages. [cited by applicant]
Chen et al., “Algorithm description for Versatile Video Coding and Test Model 5 (VTM-5),” JVET-N1002-v1 [May 21, 2019], Joint Video Experts Team (JVET), 14th Meeting: Geneva, Switzerland, Mar. 19-27, 2019 [retrieved on … [cited by applicant]
Chiang et al., “CE6-related: Latency reduction for LFNST signalling,” JVET-O0293-v5 [Jul. 9, 2019], Joint Video Experts Team (JVET) 15th Meeting: Gothenburg, Sweden, Jul. 3-12, 2019 [retrieved Jul. 9, 2019]. Retrieved f… [cited by applicant]
Extended European Search Report in EP20818014, mailed Apr. 26, 2022, 16 pages. [cited by applicant]
Office Action issued in U.S. Appl. No. 17/497,511, mailed Aug. 3, 2023, 19 pages. [cited by applicant]