IP Library Granted Patent US 12,375,717
Granted Patent B2
US 12,375,717 · App. 18/731,736 · Granted Jul 29, 2025

Mode-dependent joint component transform

Inventors: Xin Zhao (San Diego, CA); Madhu Peringassery Krishnan (Mountain View, CA); Shan Liu (San Jose, CA); Xiang Li (San Diego, CA)
Assignee: TENCENT AMERICA LLC
H04N19/60H04N19/105H04N19/159H04N19/186
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,375,717
App. No.
18/731,736
Granted
Jul 29, 2025
Kind
B2
Abstract

A method for coding video data, executable by a processor, includes receiving video data; entropy-parsing the received video data into one or more components; de-quantizing the one or more entropy-parsed components; performing a joint component secondary transformation (JCST) on the one or more components in accordance with a JCST kernel selected based on one of (i) a prediction mode corresponding to the video data, (ii) a primary transform type of a current block, and (iii) a secondary transform kernel selected for the current block; and decoding the video data based on one or more residual components corresponding to the joint component secondary transformed components.

Claims (36)

1. A method for coding video data, executable by a processor, the method comprising:

receiving video data;

entropy-parsing the received video data into one or more components;

de-quantizing the one or more entropy-parsed components;

performing a joint component secondary transformation (JCST) on the one or more components in accordance with a JCST kernel selected based on one of (i) a prediction mode corresponding to the video data, (ii) a primary transform type of a current block, and (iii) a secondary transform kernel selected for the current block; and

decoding the video data based on one or more residual components corresponding to the joint component secondary transformed components.

2. The method of claim 1 , wherein the JCST kernel is selected based on the prediction mode corresponding to the video data.

3. The method of claim 2 , wherein the JCST kernel is applied based on an intra prediction mode associated with the video data.

4. The method of claim 3 , wherein the intra prediction mode comprises one or more from among DC, SMOOTH_H, SMOOTH_V, directional modes, Paeth predictor, and filtering modes.

5. The method of claim 3 , wherein the JCST kernel is applied based on a nominal intra prediction modes.

6. The method of claim 3 , wherein JCST is disabled based on the intra prediction mode.

7. The method of claim 6 , wherein the intra prediction mode comprises one or more from among chroma-from-luma mode and Paeth Predictor.

8. The method of claim 2 , wherein different JCST kernels are applied for intra and inter prediction modes.

9. The method of claim 2 , wherein the JCST kernel is applied based on an Intra Block Copy mode.

10. The method of claim 2 , wherein the JCST kernel is applied based on a Palette mode.

11. The method of claim 2 , wherein the JCST kernel is applied based on different inter prediction modes.

12. The method of claim 2 , wherein the JCST is applied based on motion information.

13. The method of claim 12 , wherein the JCST kernels is dependent on whether a motion vector predictor comes from a top, a left, or a temporal motion vector predictor candidate.

14. The method of claim 12 , wherein the JCST kernel is dependent on a magnitude of the motion vector.

15. The method of claim 12 , wherein the JCST kernel is dependent on a direction of the motion vector.

16. The method of claim 1 , wherein the JCST kernel is selected based on the primary transform type selected for the current block associated with the video data.

17. The method of claim 16 , wherein the JCST kernel is applied based on a primary transform being applied on color components associated with the video data applying JCST.

18. The method of claim 1 , wherein the JCST kernel is selected based on the secondary transform kernel selected for the current block associated with the video data.

19. A decoder for decoding video data, the computer system comprising:

one or more computer-readable non-transitory storage media configured to store computer program code; and

one or more computer processors configured to access said computer program code and operate as instructed by said computer program code, said computer program code including:

receiving code configured to cause the one or more computer processors to receive video data;

entropy-parsing code configured to cause the one or more computer processors to entropy-parse the received video data into one or more components;

de-quantizing code configured to cause the one or more computer processors to de-quantize the one or more entropy-parsed components;

performing code configured to cause the one or more computer processors to perform a joint component secondary transformation (JCST) on the one or more components in accordance with a JCST kernel selected based on one of (i) a prediction mode corresponding to the video data, (ii) a primary transform type of a current block, and (iii) a secondary transform kernel selected for the current block; and

decoding code configured to cause the one or more computer processors to decode the video data based on one or more residual components corresponding to the joint component secondary transformed components.

20. A method for decoding a video bitstream, executable by a processor, the method comprising:

decoding the video bitstream based on one or more residual components corresponding to joint component secondary transformed components,

wherein the video bitstream is entropy-parsed into one or more components;

wherein the one or more entropy-parsed components are de-quantized;

wherein a joint component secondary transformation (JCST) is performed on the one or more de-quantized components in accordance with a JCST kernel selected based on one of (i) a prediction mode corresponding to the video data, (ii) a primary transform type of a current block, and (iii) a secondary transform kernel selected for the current block.

Continuity (4)
Continuation 17695190 · Mar 15, 2022
Continuation 17061829 · Oct 2, 2020
Provisional Application 63030623 · May 27, 2020
Related Publication 20240323440A1 · Sep 26, 2024
References Cited (47)
US 10536720B2 · Zhao et al. · 2020 [cited by applicant]
US 10567769B2 · Li et al. · 2020 [cited by applicant]
US 11310529B2 · Zhao · 2022 [cited by examiner]
US 12010343B2 · Zhao · 2024 [cited by examiner]
US 20070237240A1 · Lee et al. · 2007 [cited by applicant]
US 20160100167A1 · Rapaka et al. · 2016 [cited by applicant]
US 20160105682A1 · Rapaka et al. · 2016 [cited by applicant]
US 20160361041A1 · Barsimantov et al. · 2016 [cited by applicant]
US 20170295380A1 · Huang et al. · 2017 [cited by applicant]
US 20190208217A1 · Zhou et al. · 2019 [cited by applicant]
US 20200092583A1 · Zhao et al. · 2020 [cited by applicant]
US 20220116606A1 · Pfaff · 2022 [cited by examiner]
US 20220321916A1 · Zhu · 2022 [cited by examiner]
KR 1020180063186A · 2018 [cited by applicant]
WO 2019173522A1 · 2019 [cited by applicant]
WO 2020035827A1 · 2020 [cited by applicant]
Bross et al., “General Video Coding Technology in Responses to the Joint Call for Proposals on Video Compression with Capability beyond Hevc”, IEEE Transactions on Circuits and Systems for Video Technology, 2019, pp. 1-… [cited by applicant]
Chang et al., “Intra prediction using multiple reference lines for the versatile video coding standard”, Proc. SPIE 11137, Applications of Digital Image Processing XLII, 1113716 (Sep. 6, 2019) (8 pages total). [cited by applicant]
Liu et al., “JVET AHG report: Neural Networks in Video Coding (AHG9)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 10th Meeting: San Diego, US, Apr. 10-20, 2018, JVET-J0009-v1 (3 pa… [cited by applicant]
Bross et al., “Versatile Video Coding (Draft 2)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 11th Meeting: Ljubljana, SI, Jul. 10-18, 2018, JVET-K1001-v6 (141 pages total). [cited by applicant]
Bross et al., “CE3: Multiple reference line intra prediction (Test 1.1.1, 1.1.2, 1.1.3 and 1.1.4)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Meeting: Macao, CN, Oct. 3-12, … [cited by applicant]
Zhao et al., “CE6: On 8-bit primary transform core (Test 6.1.3)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Meeting: Macao, CN, Oct. 3-12, 2018, JVET-L0285-r1 (35 pages tota… [cited by applicant]
Zhao et al., “CE6: Fast DST-7/DCT-8 with dual implementation support (Test 6.2.3)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, JVET-M… [cited by applicant]
Li et al., “Methodology and reporting template for neural network coding tool testing”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 13th Meeting: Marrakech, MA, Jan. 9-18, 2019, JV… [cited by applicant]
Bross et al., “Versatile Video Coding (Draft 5)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 14th Meeting: Geneva, CH, Mar. 19-27, 2019, VET-N1001-v10 (407 pages total). [cited by applicant]
Galpin et al., “Non-CE4: Affine and sub-block modes coding clean-up”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, JVET-O0500_v3 (4 pa… [cited by applicant]
Zhao et al., Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, JVET-O0539-v2 (19 pages total). [cited by applicant]
Zhao et al., “Non-CE6: Configurable max transform size in VVC”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, JVET-O0545-v2 (6 pages to… [cited by applicant]
Bross et al., “Versatile Video Coding (Draft 6)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 15th Meeting: Gothenburg, SE, Jul. 3-12, 2019, JVET-O2001-vDvE (455 pages total). [cited by applicant]
Liu et al., “Joint Temporal-Spatial Bit Allocation for Video Coding With Dependency”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 15, No. 1, Jan. 2005 (12 pages total). [cited by applicant]
Liu et al., “Deep Learning-Based Technology in Responses to the Joint Call for Proposals on Video Compression with Capability beyond HEVC”, IEEE Transactions on Circuits and Systems for Video Technology, 2019 (14 pages … [cited by applicant]
Zhang et al., “Fast Adaptive Multiple Transform for Versatile Video Coding”, 2019 Data Compression Conference (DCC), IEEE, pp. 63-72 (10 pages total). [cited by applicant]
Zhao et al., “NSST: Non-Separable Secondary Transforms for Next Generation Video Coding”, 2016 Picture Coding Symposium (PCS), Dec. 4-7, 2016 (5 pages total). [cited by applicant]
Zhao et al., “Low-Complexity Intra Prediction Refinements for Video Coding”, 2018 Picture Coding Symposium (PCS), Jun. 24-27, 2018 (5 pages total). [cited by applicant]
Zhao et al., Joint Separable and Non-Separable Transforms for Next-Generation Video Coding, IEEE Transactions on Image Processing, vol. 27, Issue 5, May 2018 (13 pages total). [cited by applicant]
Zhao et al., “Coupled Primary and Secondary Transform for Next Generation Video Coding”, 2018 IEEE Visual Communications and Image Processing (VCIP), Dec. 9-12, 2018 (4 pages total). [cited by applicant]
Zhao et al., “Wide Angular Intra Prediction for Versatile Video Coding”, 2019 Data Compression Conference (DCC), IEEE 2019, pp. 53-62 (10 pages total). [cited by applicant]
Bonatto et al. “Towards an Efficient Memory Architecture for Video Decoding Systems”, IEEE, 2012, Retrieved on Jul. 30, 2021 from <https://ieeexplore.ieee.org/abstract/document/6473662>, 8 pages. [cited by applicant]
International Search Report dated Sep. 7, 2021 from the International Searching Authority in International Application No. PCT/US2021/034227. [cited by applicant]
Written Opinion dated Sep. 7, 2021 from the International Bureau in International Application No. PCT/US2021/034227. [cited by applicant]
Chen et al., “Algorithm Description of Joint Exploration Test Model 1”, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, J VET-A1001, Geneva, CH, Oct. 19-21, 2015 (27 pages total). [cited by applicant]
Saxena et al., “Low Latency Secondary Transforms for Intra/Inter Prediction Residual”, IEEE Transactions on Image Processing, vol. 22. No. 10, Oct. 2013, pp. 4061-4071 (11 pages total). [cited by applicant]
Bita et al., “On optimal transforms in lossy compression of multicomponent images with JPEG2000”, Signal Processing, vol. 90, 2010, pp. 759-773 (15 pags total). [cited by applicant]
Bayazit et al., “Adaptive Spectral Transform for Wavelet-Based Color Image Compression”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 21, No. 7, Jul. 2011(10 pages total). [cited by applicant]
Extended European Search Report date Nov. 16, 2022 from the European Patent Office in EP Application No. 21811755.4. [cited by applicant]
Rudat et al., “Inter-Component Transform for Color Video Coding”, 2019 IEEE, Picture Coding Symposium, Nov. 2019. [cited by applicant]
Office Action received in Korean Application No. 10-2022-7008213, mailed Aug. 13, 2024. [cited by applicant]