IP Library Granted Patent US 12,495,142
Granted Patent B2
US 12,495,142 · App. 18/340,395 · Granted Dec 9, 2025

Intra interpolation filter for multi-line intra prediction

Inventors: Liang Zhao (Sunnyvale, CA); Xin Zhao (San Diego, CA); Xiang Li (Los Gatos, CA); Shan Liu (San Jose, CA)
Assignee: TENCENT AMERICA LLC
H04N19/117G06T5/20G06T5/70G06T5/73H04N19/105H04N19/132H04N19/159G06T2207/20192H04N19/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,495,142
App. No.
18/340,395
Granted
Dec 9, 2025
Kind
B2
Abstract

A method and apparatus for selecting an intra interpolation filter for multi-line intra prediction based on a reference line index for decoding a video sequence includes identifying a set of reference lines associated with a coding unit. A first type of interpolation filter is applied to reference samples included in a first reference line, of the set of reference lines, that is adjacent to the coding unit to generate a first set of prediction samples based on the first reference line being associated with a first reference line index. A second type of interpolation filter is applied to reference samples included in a second reference line, of the set of reference lines, that is non-adjacent to the coding unit to generate a second set of prediction samples based on the second reference line being associated with a second reference line index.

Claims (50)

1 . A method for decoding a video sequence, comprising:

identifying a set of reference lines associated with a coding unit;

applying a first filter selected from an edge smoothing type of filter or an edge preserving type of filter to reference samples included in a first reference line of the set of reference lines based on the first reference line being associated with a first reference line index, wherein the first reference line index indicates that the first reference line is adjacent to the coding unit, and wherein the edge smoothing type of filter comprises a linear interpolation filter having only positive or zero filter coefficients; and

applying a second filter selected from only the edge preserving type of filter to reference samples included in a second reference line of the set of reference lines based on the second reference line being associated with a second reference line index, wherein the second reference line index indicates that the second reference line is non-adjacent to the coding unit, wherein the edge preserving type of filter comprises a linear interpolation filter having at least one negative filter coefficient, a non-linear interpolation filter, a Cubic filter, a discrete cosine transform (DCT)-based filter, a polynomial-based interpolation filter, a bilateral filter, and a Hermite interpolation filter.

2 . The method of claim 1 , wherein the edge smoothing type of filter comprises a bilinear filter and a Gaussian filter.

3 . The method of claim 1 , wherein applying the first filter comprises applying a first type of edge preserving filter, and wherein applying the second filter comprises applying a second type of edge preserving filter that is different than the first type of edge preserving filter.

4 . The method of claim 1 , wherein applying the first filter comprises applying an M-tap edge preserving filter, wherein applying the second filter comprises applying an N-tap interpolation filter, and wherein M is not equal to N.

5 . The method of claim 1 , wherein applying the first filter comprises applying a first edge preserving filter associated with a first set of filter coefficients, and wherein applying the second filter comprises applying a second edge preserving filter associated with a second set of filter coefficients that is different than the first set of filter coefficients.

6 . The method of claim 1 , wherein applying the first filter comprises applying an edge preserving filter including fixed filter coefficients.

7 . The method of claim 1 , wherein applying the first filter comprises applying an edge smoothing filter in association with wide angle modes of the first reference line, and wherein applying the second filter comprises applying an edge preserving filter that is applied to wide angle modes of the second reference line.

8 . The method of claim 1 , further comprising:

generating a first set of prediction samples by applying the first filter; and

generating a second set of prediction samples by applying the second filter.

9 . A method for encoding a video sequence, comprising:

identifying a set of reference lines associated with a coding unit;

selecting a first filter from an edge smoothing type of filter or an edge preserving type of filter to be applied to reference samples included in a first reference line of the set of reference lines based on the first reference line being associated with a first reference line index, wherein the first reference line index indicates that the first reference line is adjacent to the coding unit, and wherein the edge smoothing type of filter comprises a linear interpolation filter having only positive or zero filter coefficients;

generating a first set of prediction samples based on the first reference line;

selecting a second filter from only the edge preserving type of filter, not the edge smoothing type of filter, to be applied to reference samples included in a second reference line of the set of reference lines based on the second reference line being associated with a second reference line index, wherein the second reference line index indicates that the second reference line is non-adjacent to the coding unit, wherein the edge preserving type of filter comprises a linear interpolation filter having at least one negative filter coefficient, a non-linear interpolation filter, a Cubic filter, a discrete cosine transform (DCT)-based filter, a polynomial-based interpolation filter, a bilateral filter, and a Hermite interpolation filter; and

generating a second set of prediction samples based on the second reference line, wherein only most probable modes (MPMs) are allowed for the second reference line.

10 . The method of claim 9 , wherein the edge smoothing type of filter comprises a bilinear filter and a Gaussian filter.

11 . The method of claim 9 , wherein:

selecting the first filter comprises selecting a first type of edge preserving filter; and

selecting the second filter comprises selecting a second type of edge preserving filter that is different than the first type of edge preserving filter.

12 . The method of claim 9 , wherein:

applying the first filter comprises applying an M-tap edge preserving filter;

applying the second filter comprises applying an N-tap interpolation filter; and

M is not equal to N.

13 . The method of claim 9 , wherein:

selecting the first filter comprises selecting a first edge preserving filter associated with a first set of filter coefficients; and

selecting the second filter comprises selecting a second edge preserving filter associated with a second set of filter coefficients that is different than the first set of filter coefficients.

14 . The method of claim 9 , wherein selecting the first filter comprises selecting an edge preserving filter including fixed filter coefficients.

15 . The method of claim 9 , wherein:

selecting the first filter comprises selecting an edge smoothing filter in association with wide angle modes of the first reference line; and

selecting the second filter comprises selecting an edge preserving filter that is applied to wide angle modes of the second reference line.

16 . A method of processing visual media data, the method comprising:

performing a conversion between a visual media file and a bitstream of a visual media data according to a format rule,

wherein the bitstream comprises a plurality of coding units; and

wherein the format rule specifies that:

a set of reference lines associated with a coding unit of the plurality of coding units is to be identified;

a first filter, selected from an edge smoothing type of filter or an edge preserving type of filter, is to be applied to reference samples included in a first reference line of the set of reference lines based on the first reference line being associated with a first reference line index, wherein the first reference line index indicates that the first reference line is adjacent to the coding unit, and wherein the edge smoothing type of filter comprises a linear interpolation filter having only positive or zero filter coefficients; and

a second filter, selected from only the edge preserving type of filter, is to be applied to reference samples included in a second reference line of the set of reference lines based on the second reference line being associated with a second reference line index, wherein the second reference line index indicates that the second reference line is non-adjacent to the coding unit, wherein the edge preserving type of filter comprises a linear interpolation filter having at least one negative filter coefficient, a non-linear interpolation filter, a Cubic filter, a discrete cosine transform (DCT)-based filter, a polynomial-based interpolation filter, a bilateral filter, and a Hermite interpolation filter.

17 . The method of claim 16 , wherein the edge smoothing type of filter comprises a bilinear filter and a Gaussian filter.

18 . The method of claim 16 , wherein:

applying the first filter comprises applying a first type of edge preserving filter, and

applying the second filter comprises applying a second type of edge preserving filter that is different than the first type of edge preserving filter.

19 . The method of claim 16 , wherein:

applying the first filter comprises applying an M-tap edge preserving filter,

applying the second filter comprises applying an N-tap interpolation filter, and

M is not equal to N.

20 . The method of claim 16 , wherein applying the first filter comprises applying an edge preserving filter including fixed filter coefficients.

Continuity (4)
Continuation 17061044 · Oct 1, 2020
Continuation 16202902 · Nov 28, 2018
Provisional Application 62729395 · Sep 10, 2018
Related Publication 20230336720A1 · Oct 19, 2023
References Cited (46)
US 11323705B2 · Lee · 2022 [cited by examiner]
US 20080123974A1 · Zhu · 2008 [cited by examiner]
US 20120082224A1 · Van Der Auwera et al. · 2012 [cited by applicant]
US 20130003832A1 · Li · 2013 [cited by examiner]
US 20140058528A1 · Contreras-Vidal · 2014 [cited by examiner]
US 20160295243A1 · Ashrafi · 2016 [cited by examiner]
US 20160330478A1 · Jeon · 2016 [cited by examiner]
US 20170347093A1 · Yu · 2017 [cited by examiner]
US 20170353730A1 · Liu · 2017 [cited by examiner]
US 20170359595A1 · Zhang · 2017 [cited by examiner]
US 20180091825A1 · Zhao · 2018 [cited by examiner]
US 20180295361A1 · Kim · 2018 [cited by examiner]
US 20180324417A1 · Karczewicz · 2018 [cited by examiner]
US 20180332284A1 · Liu · 2018 [cited by examiner]
US 20180367814A1 · Seregin · 2018 [cited by examiner]
US 20190110045A1 · Zhao · 2019 [cited by examiner]
US 20190141318A1 · Li · 2019 [cited by examiner]
US 20190238835A1 · Lee · 2019 [cited by examiner]
US 20200007870A1 · Ramasubramonian · 2020 [cited by examiner]
US 20200007895A1 · Van der Auwera · 2020 [cited by examiner]
US 20200014922A1 · Ramasubramonian · 2020 [cited by examiner]
US 20210120240A1 · Bross · 2021 [cited by examiner]
US 20240314306A1 · Lee · 2024 [cited by examiner]
EA 3477951A1 · 2019 [cited by applicant]
EP 3477951A1 · 2019 [cited by examiner]
EP 18180960A1 · 2019 [cited by examiner]
WO WO2017190288A1 · 2017 [cited by examiner]
WO WO2017222326A1 · 2017 [cited by examiner]
Chen et al. Algorithm description for Versatile Video Coding and Test Model 1 (VTM 1), Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 10th Meeting: San Diego, US, Apr. 10-20, 2018. (Y… [cited by examiner]
Chen J. and E. Alshina Algorithm description for Versatile Video Coding and Test Model 1 (VTM 1), Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IECJTC 1/SC 29/WG 11, 10th Meeting: San Diego, US, Apr. 10-20… [cited by examiner]
Communication dated Dec. 10, 2020, from the European Patent Office in application No. 19861175.8. [cited by applicant]
Geert Van der Auwera et al., “Description of Core Experiment 3: Intra Prediction and Mode Coding”, JVET of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-J1023_r2, Apr. 10-20, 2018 (50 pages total). [cited by applicant]
Hahyun Lee et al., “ETRI (Electronics and Telecommunications Research Institute)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Jul. 10-18, 2018, ( 4 pages total). [cited by applicant]
International Search Report from the International Bureau in counterpart International Application No. PCT/US2019/047011, dated Sep. 24, 2019. [cited by applicant]
Jianle Chen et al., “Algorithm Description of Joint Exploration Test Model 6 (JEM 6)”, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, Document: JVET-F1001-v3, 6th Meeting: Hobart,… [cited by applicant]
Jung Won Kang et al., “Description of SDR video coding technology proposal by ETRI and Sejong University”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-J0013-v1, Apr. 10-20, 20… [cited by applicant]
Liang Zhao et al., “CE3-related: MPM based multi-line intra prediction scheme”, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 11th Meeting: Ljubljana, SI, Oct. 18, 2018, JVET-K04… [cited by applicant]
Notice of Reasons for Refusal dated Dec. 14, 2021 from the Japanese Patent Office in JP Application No. 2020-561054. [cited by applicant]
Office Action dated Jul. 15, 2022 issued by the Korean Patent Office in Korean Application No. 10-2020-7031582. [cited by applicant]
Written Opinion from the International Bureau in counterpart International Application No. PCT/US2019/047011, dated Sep. 24, 2019. [cited by applicant]
Yi Wen Chen et al., “Description of SDR, HDR and 360° video coding technology proposal by Qualcomm and Technicolor—low and high complexity versions”, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC J… [cited by applicant]
Kang et al. Joint Video ExpertsTeam (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 10th Meeting: San Diego, US, Apr. 10-20, 2018 (Year: 2018). [cited by applicant]
Jiang et al. Joint Video ExpertsTeam (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 11th Meeting Ljubljana, SI, Jul. 10-18, 2018, CE3-related: Advanced MPM based on intra reference line selection scheme (Year:… [cited by applicant]
Jinho Lee et al., “CE3: Multi-line based intra prediction (Test 5.3.1, 5.3.2, 5.3.3)”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-K0166-v1, Jul. 10-18, 2018 (4 pages total). [cited by applicant]
Tencent America LLC, Japanese Office Action, JP Patent Application No. 2024-103519, Jun. 10, 2025, 16 pgs. [cited by applicant]
M. Albrecht et al., “Description of SDR, HDR, and 360° Video Coding Technology Proposal by Fraunhofer HHI”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 10th Meeting: San Diego, US,… [cited by applicant]