IP Library Granted Patent US 12,231,648
Granted Patent B2
US 12,231,648 · App. 18/417,274 · Granted Feb 18, 2025

Methods, architectures, apparatuses and systems directed to improved linear model estimation for template based video coding

Inventors: Saurav Bandyopadhyay (San Diego, CA); Xiaoyu Xiu (San Diego, CA); Yuwen He (San Diego, CA)
Assignee: InterDigital VC Holdings, Inc.
H04N19/132H04N19/105H04N19/176H04N19/186
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,231,648
App. No.
18/417,274
Granted
Feb 18, 2025
Kind
B2
Abstract

Procedures, methods, architectures, apparatuses, systems, devices, and computer program products directed to improved linear model estimation for template-based video coding are provided. Included therein is a method comprising determining minimum and maximum (“min/max”) values of luma and chroma samples neighboring a coding block, wherein the min/max chroma values correspond to the min/max luma values; determining a first linear model parameter of a template-based video coding technique (i) based on a single look-up table and the min/max chroma values and (ii) at a precision no greater than 16 bits; determining a second linear model parameter of the template-based video coding technique (i) based on the first linear model parameter and the minimum chroma and luma values and (ii) at a precision no greater than 16 bits; and predicting chroma samples of the coding block based on reconstructed luma samples of the coding block and the first and second linear model parameters.

Claims (52)

1. A method for encoding video data by an encoder, the method comprising:

coding a video block of the video data using a linear model of a template-based video coding technique, the coding comprises:

using a single look up table to determine a first parameter of the linear model, the single look up table including values for determining least significant bits, wherein the determining of the first parameter of the linear model comprises:

determining minimum and maximum values derived from luma samples neighboring the video block;

determining minimum and maximum values derived from chroma samples neighboring the video block; and

computing the first parameter of the linear model as a function of (i) the minimum and the maximum values of the chroma samples and (ii) a reciprocal of a difference between the minimum and the maximum values of the luma samples, wherein the reciprocal is derived from the single look up table, and

predicting samples of the video block based on the linear model.

2. The method according to claim 1 , further comprising:

computing a second parameter of the linear model as a function of the minimum value of the chroma samples, the minimum value of the luma samples, and the first parameter of the linear model.

3. The method according to claim 2 , wherein the second parameter is computed at a precision that is less than or equal to 16 bits.

4. The method according to claim 1 , wherein the first parameter is computed at a precision that is less than or equal to 16 bits.

5. The method according to claim 1 , wherein the template-based video coding technique is based on a cross component linear model (CCLM) or a local illumination compensation (LIC).

6. A method for decoding video data by a decoder, the method comprising:

decoding a video block of the video data using a linear model of a template-based video coding technique, the decoding comprises:

using a single look up table to determine a first parameter of the linear model, the single look up table including values for determining least significant bits, wherein the determining of the first parameter of the linear model comprises:

determining minimum and maximum values derived from luma samples neighboring the video block;

determining minimum and maximum values derived from chroma samples neighboring the video block; and

computing the first parameter of the linear model as a function of (i) the minimum and the maximum values of the chroma samples and (ii) a reciprocal of a difference between the minimum and the maximum values of the luma samples, wherein the reciprocal is derived from the single look up table, and

predicting samples of the video block based on the linear model.

7. The method according to claim 6 , further comprising:

computing a second parameter of the linear model as a function of the minimum value of the chroma samples, the minimum value of the luma samples, and the first parameter of the linear model.

8. The method according to claim 7 , wherein the second parameter is computed at a precision that is less than or equal to 16 bits.

9. The method according to claim 6 , wherein the first parameter is computed at a precision that is less than or equal to 16 bits.

10. The method according to claim 6 , wherein the template-based video coding technique is based on a cross component linear model (CCLM) or a local illumination compensation (LIC).

11. An encoder for encoding video data, comprising:

at least one processor; and

memory storing instructions that, when executed by the at least one processor, cause the encoder to:

code a video block of the video data using a linear model of a template-based video coding technique, the coding comprises:

using a single look up table to determine a first parameter of the linear model, the single look up table including values for determining least significant bits, wherein the determining of the first parameter of the linear model comprises:

determining minimum and maximum values derived from luma samples neighboring the video block;

determining minimum and maximum values derived from chroma samples neighboring the video block; and

computing the first parameter of the linear model as a function of (i) the minimum and the maximum values of the chroma samples and (ii) a reciprocal of a difference between the minimum and the maximum values of the luma samples, wherein the reciprocal is derived from the single look up table, and

predicting samples of the video block based on the linear model.

12. The encoder according to claim 11 , wherein the instructions further cause the encoder to:

compute a second parameter of the linear model as a function of the minimum value of the chroma samples, the minimum value of the luma samples, and the first parameter of the linear model.

13. The encoder according to claim 12 , wherein the second parameter is computed at a precision that is less than or equal to 16 bits.

14. The encoder according to claim 11 , wherein the first parameter is computed at a precision that is less than or equal to 16 bits.

15. The encoder according to claim 11 , wherein the template-based video coding technique is based on a cross component linear model (CCLM) or a local illumination compensation (LIC).

16. A decoder for decoding video data, comprising:

at least one processor; and

memory storing instructions that, when executed by the at least one processor, cause the decoder to:

decode a video block of the video data using a linear model of a template-based video coding technique, the decoding comprises:

using a single look up table to determine a first parameter of the linear model, the single look up table including values for determining least significant bits, wherein the determining of the first parameter of the linear model comprises:

determining minimum and maximum values derived from luma samples neighboring the video block;

determining minimum and maximum values derived from chroma samples neighboring the video block; and

computing the first parameter of the linear model as a function of (i) the minimum and the maximum values of the chroma samples and (ii) a reciprocal of a difference between the minimum and the maximum values of the luma samples, wherein the reciprocal is derived from the single look up table, and

predicting samples of the video block based on the linear model.

17. The decoder according to claim 16 , wherein the instructions further cause the decoder to:

compute a second parameter of the linear model as a function of the minimum value of the chroma samples, the minimum value of the luma samples, and the first parameter of the linear model.

18. The decoder according to claim 17 , wherein the second parameter is computed at a precision that is less than or equal to 16 bits.

19. The decoder according to claim 16 , wherein the first parameter is computed at a precision that is less than or equal to 16 bits.

20. The decoder according to claim 16 , wherein the template-based video coding technique is based on a cross component linear model (CCLM) or a local illumination compensation (LIC).

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2024
From: VID SCALE, INC.
To: INTERDIGITAL VC HOLDINGS, INC.
Reel/Frame 068284/0031 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 22, 2024
From: XIU, XIAOYU
To: VID SCALE, INC.
Reel/Frame 066193/0809 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 22, 2024
From: BANDYOPADHYAY, SAURAV; HE, YUWEN
To: VID SCALE, INC.
Reel/Frame 066193/0889 →
Continuity (3)
Continuation 17415176
Provisional Application 62784341 · Dec 21, 2018
Related Publication 20240171748A1 · May 23, 2024
References Cited (47)
US 20140294078A1 · Seregin et al. · 2014 [cited by applicant]
US 20150373372A1 · He · 2015 [cited by examiner]
US 20180077426A1 · Zhang · 2018 [cited by examiner]
US 20200154115A1 · Ramasubramonian · 2020 [cited by examiner]
US 20200195959A1 · Zhang · 2020 [cited by examiner]
US 20200195976A1 · Zhao · 2020 [cited by examiner]
US 20210258572A1 · Zhang · 2021 [cited by examiner]
US 20210297656A1 · Ma · 2021 [cited by examiner]
US 20220053192A1 · Zhang · 2022 [cited by examiner]
CN 102067543A · 2011 [cited by applicant]
CN 103891279A · 2014 [cited by applicant]
CN 104255033A · 2014 [cited by applicant]
CN 104471946A · 2015 [cited by applicant]
CN 105075259A · 2015 [cited by applicant]
CN 106464877A · 2017 [cited by applicant]
CN 106664410A · 2017 [cited by applicant]
CN 106717004A · 2017 [cited by applicant]
WO WO2018194190A1 · 2018 [cited by applicant]
WO WO2019162118A1 · 2019 [cited by examiner]
WO WO2020108591A1 · 2020 [cited by examiner]
“Enhanced Cross-Component Linear Model for Chroma Intra-Prediction in Video Coding”—Kai Zhang, Jianle Chen, Li Zhang, Xiang Li, Marta Karczewicz; IEEE Transactions on Image Processing, vol. 27, No. 8, Aug. 2018. (Year: … [cited by examiner]
“Multi-model Based Cross-component Linear Model Chroma Intra-prediction for Video Coding”—Kai Zhang, Jianle Chen, Li Zhang, Xiang Li, Marta Karczewicz; IEEE VCIP 2017, Dec. 10-13, 2017, St Petersburg, U.S.A. (Year: 2017… [cited by examiner]
Karczewicz et al., “Report of AHG1 on Coding Efficiency Improvements”, VCEG-AZ01, Qualcomm, Samsung, ITU—Telecommunications Standardization Sector, Study Group 16 Question 6, Video Coding Experts Group (VCEG), 52nd Meet… [cited by applicant]
Chen et al., “Coding Tools Investigation for Next Generation Video Coding”, ITU-T SG16, COM 16-C 806, Feb. 2015, 7 pages. [cited by applicant]
“VTM-3.0 Reference Software”, available at <https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/tags/VTM-3.0>, pp. 1-2. [cited by applicant]
Ma et al., “CE3-related: CCLM Coefficients derivation method without down-sampling operations”, JVET Meeting, Oct. 3-12, 2018, Macao, No. JVET-L0341, Oct. 7, 2018, 4 pages. [cited by applicant]
Segall et al., “Joint Call for Proposals on Video Compression with Capability beyond HEVC”, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Doc. JVET-H1002 (v6), 8th Meeting: Macao… [cited by applicant]
SMPTE 421M, “VC-1 Compressed Video Bitstream Format and Decoding Process”, SMPTE Standard, Approved Feb. 24, 2006, 493 pages. [cited by applicant]
Zhang, et al., Enhanced Cross-Component Linear Model for Chroma Intra-Prediction in Video Coding, IEEE Transactions on Image Processing, vol. 27, No. 8, pp. 3983-3997, Aug. 2018. [cited by applicant]
Bross, “Versatile Video Coding (Draft 1)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document: JVET-J1001-v2, 10th Meeting: San Diego, California, United States, Apr. 10, 2018, 4… [cited by applicant]
Alshina et. al., “Known Tools Performance Investigation for Next Generation Video Coding”, ITU—Telecommunications Standardization Sector, Video Coding Experts Group (VCEG), SG16/Q6, Power Point Presentation, VCEG-AZ05, … [cited by applicant]
Schwarz et al., “CE7: Transform Coefficient Coding and Dependent Quantization (Tests 7.1.2, 7.2.1)”, JVET of ITU-T SG16 WP3 and ISO/IEC JTC1/SC/29/WG11, 11th Meeting, Ljubljana, SI, Jul. 10-18, 2018, Document: JVET-K007… [cited by applicant]
ITU-T Recommendation H.264, “Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video, Advanced video coding for generic audiovisual services”, Telecommunication Standar… [cited by applicant]
Chen et al., “Algorithm Description of Joint Exploration Test Model 7 (JEM 7)”, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document JVET-G1001-v1, 7th Meeting: Torino, Italy, … [cited by applicant]
“BMS-2.0 Reference Software”, available at <https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_BMS/tags/BMS-2.1rc1>, 1 page. [cited by applicant]
“JEM-7.0 Reference Software”, Available at <https://jvet.hhi.fraunhofer.de/svn/svn_HMJEMSoftware/tags/HM-16.6-JEM-7.0>, 1 page. [cited by applicant]
Laroche et al., “CE3-5.1: On cross-component linear model simplification”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 12th Meeting, Macao, CN, Oct. 3-12, 2018, Document: JVET-L019… [cited by applicant]
Laroche et al., CE3: Cross-Component Linear Model Simplification (Test 5.1), 12th JVET Meeting, Oct. 3, 2018-Oct. 12, 2018, Macao, (The Joint Video Exploration Team of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16), No. JVET-L… [cited by applicant]
Tourapis et al., “H.264/14496-10 Avc Reference Software Manual”, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG01 and ITU-T SG16 Q.6), Document JVT-AE010, 31st Meeting, London, United Kingdom,… [cited by applicant]
Bross et al., “High Efficiency Video Coding (HEVC) text specification draft 10 (For FDIS & Consent)”, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document: JCTVC-… [cited by applicant]
Francois et al., “CE3-related: Reducing the number of reference samples and table size in LM Chroma process”, 125. MPEG Meeting, Jan. 14-18, 2019, Marrakech; (MPEG or ISO/IEC JTC1/SC29/WG11), No. m45367, Jan. 2, 2019, 1… [cited by applicant]
“VTM-2.0.1 reference software”, available at https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/tags/VTM-2.0.1. [cited by applicant]
Ohm et al., “Report of AHG on Future Video Coding Standardization Challenges”, International Organisation for Standardisation Organisation Internationale De Normalisation ISO/IEC JTC1/SC29/WG11 Coding of Moving Pictures… [cited by applicant]
“HM Reference Software HM-16.9”, available at <https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware>, Mar. 2016, 2 pages. [cited by applicant]
Bordes et al., “CE4-related: LIC with reduced memory buffer”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IED JTC 1/SC 29/WG 11, Document: JVET-L0203-v3, 12th Meeting, Macao, China, Oct. 3, 2018, 6 pages. [cited by applicant]
Chen et al., “Chroma intra prediction by scale luma samples using integer operations”, 3. JCT-VC Meeting, 84. MPEG Meeting Oct. 7-15, 2010, Guangzhou, JCT on VC of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.16, No. JCTVC-C206,… [cited by applicant]
Sullivan et al., “Overview of the High Efficiency Video Coding (HEVC) Standard”, Institute of Electrical and Electronics Engineers (IEEE), IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, No. 12,… [cited by applicant]