IP Library Granted Patent US 12,659,465
Granted Patent B2
US 12,659,465 · App. 18/389,665 · Granted Jun 16, 2026

Intra-prediction mode concept for block-wise picture coding

Inventors: Jonathan Pfaff (Berlin, DE); Philipp Helle (Berlin, DE); Dominique Maniry (Berlin, DE); Thomas Wiegand (Berlin, DE); Wojciech Samek (Berlin, DE); Stephan Kaltenstadler (Berlin, DE); Heiko Schwarz (Berlin, DE); Detlev Marpe (Berlin, DE); Mischa Siekmann (Berlin, DE); Martin Winken (Berlin, DE)
Assignee: Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
H04N19/109H04N19/176H04N19/61
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,659,465
App. No.
18/389,665
Granted
Jun 16, 2026
Kind
B2
Abstract

In accordance with a first aspect, an improved compression efficiency is achieved by letting a block-wise picture codec support a set of intra-prediction modes according to which the intra-prediction signal for a current block of a picture is determined by applying a set of neighboring samples of the current block onto a neural network. A second aspect of the present application is that, additionally or alternatively to the spending of neural network-based intra-prediction modes, the mode selection may be rendered more effective by the usage of a neural network dedicated to determine a rank or a probability value for each of the set of intra-prediction modes by applying a set of neighboring samples thereonto with the rank or probability value being used for the selection of one intra-prediction mode out of the plurality of intra-prediction modes including or coinciding with the set of intra-prediction modes.

Claims (69)

1 . A decoder configured to:

decode, from a data stream, a flag associated with a block, the flag having a first value specifies an intra-prediction mode for predicting the block is only from a first set of intra-prediction modes or having a second value specifies the intra-prediction mode for predicting the block is from a second set of intra-prediction modes, wherein:

the first set of intra-prediction modes does not include any angular modes, DC modes, or planar modes, and

the second set of intra-prediction modes includes a DC mode and at least one angular mode; and does not include any intra-prediction modes included in the first set of intra-prediction modes;

determine that the flag is the first value specifying that the intra-prediction mode for predicting the block is only from the first set of intra-prediction modes;

decode, from the data stream, an index indicating the intra-prediction mode from the first set of intra-prediction modes; and

predict the block using the intra-prediction mode.

2 . The decoder of claim 1 , further configured to:

determine an intra-prediction signal for the block by using a set of samples neighboring the block and the intra-prediction mode.

3 . The decoder of claim 1 , further configured to:

decode the index using a variable length code; and

select the intra-prediction mode from the first set of intra-prediction modes based on the index.

4 . The decoder of claim 1 , further configured to:

identify a set of samples neighboring the block for predicting the block,

wherein in response to the flag indicating that the intra-prediction mode for predicting the block is from the second set of intra-prediction modes, the set of neighboring samples are a one sample wide line extending alongside a left side and an upper side of the block; and

wherein in response to the flag indicating that the intra-prediction mode for predicting the block is only from the first set of intra-prediction modes, the set of neighboring samples is at least two samples wide and forms an L-shape extending along the left side and the upper side of the block.

5 . A method for decoding a picture from a data stream, the method comprising:

decoding, from the data stream, a flag associated with a block of the picture, the flag having a first value specifies an intra-prediction mode for predicting the block is only from a first set of intra-prediction modes or having a second value specifies the intra-prediction mode for predicting the block is from a second set of intra-prediction modes, wherein:

the first set of intra-prediction modes does not include any angular modes, DC modes, or planar modes, and

the second set of intra-prediction modes includes a DC mode and at least one angular mode; and does not include any intra-prediction modes included in the first set of intra-prediction modes;

determining that the flag is the first value specifying that the intra-prediction mode for predicting the block is only from the first set of intra-prediction modes;

decoding, from the data stream, an index indicating the intra-prediction mode from the first set of intra-prediction modes; and

predicting the block using the intra-prediction mode.

6 . The method of claim 5 , further comprising:

determining an intra-prediction signal for the block by using a set of samples neighboring the block and the intra-prediction mode.

7 . The method of claim 5 , further comprising:

decoding the index using a variable length code; and

selecting the intra-prediction mode from the first set of intra-prediction modes based on the index.

8 . The method of claim 5 , further comprising:

identifying a set of samples neighboring the block for predicting the block,

wherein in response to the flag indicating that the intra-prediction mode for predicting the block is from the second set of intra-prediction modes, the set of neighboring samples is a one sample wide line extending alongside a left side and an upper side of the block; and

wherein in response to the flag indicating that the intra-prediction mode for predicting the block is only from the first set of intra-prediction modes, the set of neighboring samples is at least two samples wide and forms an L-shape extending along the left side and the upper side of the block.

9 . A non-transitory, computer-readable medium storing instructions that, when executed by a processor of an electronic device, cause the electronic device to:

decode, from a data stream, a flag associated with a block, the flag having a first value specifies an intra-prediction mode for predicting the block is only from a first set of intra-prediction modes or having a second value specifies the intra-prediction mode for predicting the block is from a second set of intra-prediction modes, wherein:

the first set of intra-prediction modes does not include any angular modes, DC modes, or planar modes, and

the second set of intra-prediction modes includes a DC mode and at least one angular mode; and does not include any intra-prediction modes included in the first set of intra-prediction modes;

determine that the flag is the first value specifying that the intra-prediction mode for predicting the block is only from the first set of intra-prediction modes;

decode, from the data stream, an index indicating the intra-prediction mode from the first set of intra-prediction modes; and

predict the block using the intra-prediction mode.

10 . The non-transitory, computer-readable medium of claim 9 , further containing instructions that when executed cause the processor to:

determine an intra-prediction signal for the block by using a set of samples neighboring the block and the intra-prediction mode.

11 . The non-transitory, computer-readable medium of claim 9 , further containing instructions that when executed cause the processor to:

decode the index using a variable length code; and

select the intra-prediction mode from the first set of intra-prediction modes based on the index.

12 . The non-transitory, computer-readable medium of claim 9 , further containing instructions that when executed cause the processor to:

identify a set of samples neighboring the block for predicting the block,

wherein in response to the flag indicating that the intra-prediction mode for predicting the block is from the second set of intra-prediction modes, the set of neighboring samples is a one sample wide line extending alongside a left side and an upper side of the block; and

wherein in response to the flag indicating that the intra-prediction mode for predicting the block is only from the first set of intra-prediction modes, the set of neighboring samples is at least two samples wide and forms an L-shape extending along the left side and the upper side of the block.

13 . An encoder configured to:

determine whether an intra-prediction mode for encoding a block is from a first set of intra-prediction modes or a second set of intra-prediction modes, wherein:

the first set of intra-prediction modes does not include any angular modes, DC modes, or planar modes, and

the second set of intra-prediction modes includes a DC mode and at least one angular mode; and does not include any intra-prediction modes included in the first set of intra-prediction modes;

encode a flag associated with the block, the flag having a first value or a second value, the first value specifying that the determined intra-prediction mode for encoding the block is only from the first set of intra-prediction modes, the second value specifying that the determined intra-prediction mode for encoding the block is from the second set of intra-prediction modes;

in response to the determination that the intra-prediction mode for encoding the block is from the first set of intra-prediction modes, encode an index indicating the intra-prediction mode from the first set of intra-prediction modes; and

encode the block using the intra-prediction mode from the first set of intra-prediction modes when the flag has the first value.

14 . A method for encoding a picture, the method comprising:

determining whether an intra-prediction mode for encoding a block of the picture is from a first set of intra-prediction modes or a second set of intra-prediction modes, wherein:

the first set of intra-prediction modes does not include any angular modes, DC modes, or planar modes, and

the second set of intra-prediction modes includes a DC mode and at least one angular mode; and does not include any intra-prediction modes included in the first set of intra-prediction modes;

encoding a flag associated with the block, the flag having a first value or a second value, the first value specifying that the determined intra-prediction mode for encoding the block is only from the first set of intra-prediction modes, the second value specifying that the determined intra-prediction mode for encoding the block is from the second set of intra-prediction modes;

in response to determining that the intra-prediction mode for encoding the block is from the first set of intra-prediction modes, encoding an index indicating the intra-prediction mode from the first set of intra-prediction modes; and

encoding the block using the intra-prediction mode from the first set of intra-prediction modes when the flag has the first value.

15 . A non-transitory, computer-readable medium storing instructions that, when executed by a processor of an electronic device, cause the electronic device to:

determine whether an intra-prediction mode for encoding a block is from a first set of intra-prediction modes or a second set of intra-prediction modes, wherein:

the first set of intra-prediction modes does not include any angular modes, DC modes, or planar modes, and

the second set of intra-prediction modes includes a DC mode and at least one angular mode; and does not include any intra-prediction modes included in the first set of intra-prediction modes;

encode a flag associated with the block, the flag having a first or a second value, the first value specifying that the determined intra-prediction mode for encoding the block is only from the first set of intra-prediction modes, the second value specifying that the determined intra-prediction mode for encoding the block is from the second set of intra-prediction modes;

in response to the determination that the intra-prediction mode for encoding the block is from the first set of intra-prediction modes, encode an index indicating the intra-prediction mode from the first set of intra-prediction modes; and

encode the block using the intra-prediction mode from the first set of intra-prediction modes when the flag has the first value.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 2, 2025
From: PFAFF, JONATHAN; HELLE, PHILIPP; MANIRY, DOMINIQUE; WIEGAND, THOMAS; SAMEK, WOJCIECH; KALTENSTADLER, STEPHAN; SCHWARZ, HEIKO; MARPE, DETLEV; SIEKMANN, MISCHA; WINKEN, MARTIN
To: FRAUNHOFER-GESELLSCHAFT ZUR FÖRDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
Reel/Frame 071789/0550 →
Priority Claims (1)
EP 17196402 · Oct 13, 2017 · regional
Continuity (4)
Continuation 17839459 · Jun 13, 2022
Continuation 16845715 · Apr 10, 2020
Continuation PCTEP2018077609 · Oct 10, 2018
Related Publication 20240137500A1 · Apr 25, 2024
References Cited (23)
US 10171813B2 · Zheng et al. · 2019 [cited by applicant]
US 11197013B2 · Lee et al. · 2021 [cited by applicant]
US 20090110070A1 · Takahashi et al. · 2009 [cited by applicant]
US 20120327999A1 · Francois et al. · 2012 [cited by applicant]
US 20130016773A1 · Oh et al. · 2013 [cited by applicant]
US 20130279582A1 · Sugimoto · 2013 [cited by examiner]
US 20150023406A1 · Lee et al. · 2015 [cited by applicant]
US 20180184123A1 · Terada · 2018 [cited by examiner]
US 20200236349A1 · Zhai et al. · 2020 [cited by applicant]
US 20210127140A1 · Miska et al. · 2021 [cited by applicant]
US 20210218997A1 · Rezazadegan Tavakoli et al. · 2021 [cited by applicant]
CN 111901592A · 2020 [cited by applicant]
EP 2063644A2 · 2009 [cited by applicant]
EP 3310058A1 · 2018 [cited by applicant]
JP 2009111691A · 2009 [cited by applicant]
JP 2013012791A · 2013 [cited by applicant]
WO 2016199330A1 · 2016 [cited by applicant]
Cui, Wenzue, et al.; “Convolutional Neural Networks Based Intra Prediction for HEVC”; Research Gate Conference Paper; Apr. 2017; 11 pages. [cited by applicant]
Li, Jiahao, et al.; “Intra Prediction Fully Connected Network For Video Coding”; IEEE; 2017; 5 pages. [cited by applicant]
Pfaff, Jonathan et al., “Intra prediction modes based on neural networks”, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC1/SC 29/WG 11 10th Meeting: San Diego, US, Apr. 10-20, 2018, [JVET-J0037-v1],… [cited by applicant]
Sun et al. (Enhanced Intra Prediction for Video Coding by Using Multiple Neural Networks), IEEE vol. 22 No. 11, Nov. 2020. [cited by applicant]
Toyozaki, Tomohiro et al., “Fast Intra Prediction Mode Decision Method for HEVC Coding by using Deep Learning in HEVC Intra Coding”, Proceedings of the 2016 ITE Winter Annual Convention Proceedings of the 2016 ITE Winte… [cited by applicant]
Wein “Intra Prediction” HEVC Jan. 10, 2015. Retrieved from the Internet: URL:https://www.hevcbook.de/intra-prediction/* the whole document *. [cited by applicant]