IP Library Granted Patent US 11,601,672
Granted Patent B2
US 11,601,672 · App. 17/032,113 · Granted Mar 7, 2023

Intra-prediction mode concept for block-wise picture coding

Inventors: Jonathan Pfaff (Berlin, DE); Philipp Helle (Berlin, DE); Philipp Merkle (Berlin, DE); Björn Stallenberger (Berlin, DE); Mischa Siekmann (Berlin, DE); Martin Winken (Berlin, DE); Adam Wieckowski (Berlin, DE); Wojciech Samek (Berlin, DE); Stephan Kaltenstadler (Berlin, DE); Heiko Schwarz (Berlin, DE); Detlev Marpe (Berlin, DE); Thomas Wiegand (Berlin, DE)
Assignee: Fraunhofer-Gesellschaft zur Forderung der angewandten Forschung e.V.
H04N19/593H04N19/105H04N19/11H04N19/176H04N19/61
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,601,672
App. No.
17/032,113
Granted
Mar 7, 2023
Kind
B2
Abstract

An apparatus for block-wise decoding a picture from a data stream and/or encoding a picture into a data stream, the apparatus supporting at least one intra-prediction mode according to which the intra-prediction signal for a block of a predetermined size of the picture is determined by applying a first template of samples which neighbours the current block onto a neural network. The apparatus may be configured, for a current block differing from the predetermined size, to: resample a second template of samples neighboring the current block, so as to conform with the first template so as to obtain a resampled template; apply the resampled template of samples onto the neural network so as to obtain a preliminary intra-prediction signal; and resample the preliminary intra-prediction signal so as to conform with the current block so as to obtain the intra-prediction signal for the current block.

Claims (103)

1. An apparatus for block-wise decoding a picture from a data stream, the apparatus supporting at least one intra-prediction mode according to which an intra-prediction signal for a block of a predetermined size of the picture is determined by applying a first template of samples which neighbors the block, wherein the apparatus comprises:

a processor; and

a memory storing instructions that, when executed by the processor, cause the processor, for a current block differing from the predetermined size, to:

resample a second template of already reconstructed samples neighboring the current block, so as to acquire a resampled template with dimensions of the first template, by downsampling the second template to acquire the resampled template;

perform an intra prediction by applying the resampled template of samples so as to acquire a preliminary intra-prediction signal; and

resample the preliminary intra-prediction signal so as to have a dimension of the current block so as to acquire the intra-prediction signal for the current block, wherein resampling the preliminary intra-prediction signal includes upsampling the preliminary intra-prediction signal.

2. The apparatus of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to apply the resampled template of samples onto a neural network so as to acquire the preliminary intra-prediction signal.

3. The apparatus of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to:

transform the preliminary intra-prediction signal from a spatial domain into a transform-domain; and

resample the preliminary intra-prediction signal in the transform-domain.

4. The apparatus of claim 3 , wherein the instructions, when executed by the processor, further cause the processor to:

resample the transform-domain preliminary intra-prediction signal by scaling coefficients of the preliminary intra-prediction signal.

5. The apparatus of claim 3 , wherein the instructions, when executed by the processor, further cause the processor to:

resample the transform-domain preliminary intra-prediction signal by:

increasing dimensions of the intra-prediction signal to comprise the dimensions of the current block; and

zero-padding coefficients of added coefficients of the preliminary intra-prediction signal, the added coefficients of higher-frequencies bins.

6. The apparatus of claim 3 , wherein the instructions, when executed by the processor, further cause the processor to:

add the transform-domain preliminary intra-prediction signal to a dequantized version of a prediction residual signal.

7. The apparatus of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to:

resample the preliminary intra-prediction signal in a spatial domain.

8. The apparatus of claim 7 , wherein the instructions, when executed by the processor, further cause the processor to:

resample the preliminary intra-prediction signal by performing a bilinear interpolation.

9. The apparatus of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to:

decode in a data field information regarding at least one of (i) the resampling and (ii) a utilization of neural networks for different dimensions.

10. The apparatus of claim 1 , the apparatus supporting at least one intra-prediction mode according to which the intra-prediction signal for the current block of the picture is determined by:

applying a first set of neighboring samples of the current block onto a neural network to acquire a prediction of a set of transform coefficients of a transform of the current block.

11. The apparatus of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to:

inversely transform the prediction to obtain a reconstructed signal.

12. The apparatus of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to:

decode from the data stream an index using a variable length code; and

perform a selection using the index.

13. The apparatus of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to:

determine a ranking of a set of intra prediction modes; and, subsequently, resample the second template.

14. An apparatus for block-wise encoding a picture into a data stream, the apparatus supporting at least one intra-prediction mode according to which an intra-prediction signal for a block of a predetermined size of the picture is determined by applying a first template of samples which neighbors the block, wherein the apparatus comprises:

a processor; and

a memory storing instructions that, when executed by the processor, cause the processor, for a current block differing from the predetermined size, to:

resample a second template of already reconstructed samples neighboring the current block so as to acquire a resampled template with dimensions of the first template, by downsampling the second template to acquire the resampled template;

perform an intra prediction by applying the resampled template of samples so as to acquire a preliminary intra-prediction signal; and

resample the preliminary intra-prediction signal so as to have a dimension of the current block so as to acquire the intra-prediction signal for the current block, wherein resampling the preliminary intra-prediction signal includes upsampling the preliminary intra-prediction signal.

15. The apparatus of claim 14 , wherein the instructions, when executed by the processor, further cause the processor to apply the resampled template of samples onto a neural network so as to acquire the preliminary intra-prediction signal.

16. The apparatus of claim 14 , wherein the instructions, when executed by the processor, further cause the processor to:

transform the preliminary intra-prediction signal from a spatial domain into a transform-domain; and

resample the preliminary intra-prediction signal in the transform-domain.

17. The apparatus of claim 16 , wherein the instructions, when executed by the processor, further cause the processor to:

resample the transform-domain preliminary intra-prediction signal by scaling coefficients of the preliminary intra-prediction signal.

18. The apparatus of claim 16 , wherein the instructions, when executed by the processor, further cause the processor to:

resample the transform-domain preliminary intra-prediction signal by:

increasing dimensions of the intra-prediction signal to comprise the dimensions of the current block; and

zero-padding coefficients of added coefficients of the preliminary intra-prediction signal, the added coefficients relating to higher-frequencies bins.

19. The apparatus of claim 16 wherein the instructions, when executed by the processor, further cause the processor to:

add the transform-domain preliminary intra-prediction signal to a dequantized version of a prediction residual signal.

20. The apparatus of claim 14 , wherein the instructions, when executed by the processor, further cause the processor to:

resample the preliminary intra-prediction signal in a spatial domain.

21. The apparatus of claim 20 , wherein the instructions, when executed by the processor, further cause the processor to:

resample the preliminary intra-prediction signal by performing a bilinear interpolation.

22. The apparatus of claim 14 , wherein the instructions, when executed by the processor, further cause the processor to:

encode in a data field information regarding at least one of (i) the resampling and (ii) a utilization of neural networks for different dimensions.

23. The apparatus of claim 14 , the apparatus supporting at least one intra-prediction mode according to which the intra-prediction signal for the current block of the picture is determined by:

applying a first set of neighboring samples of the current block onto a neural network to acquire a prediction of a set of transform coefficients of a transform of the current block.

24. The apparatus of claim 14 , wherein the instructions, when executed by the processor, further cause the processor to:

inversely transform the prediction to acquire a reconstructed signal.

25. The apparatus of claim 14 , wherein the instructions, when executed by the processor, further cause the processor to:

decode from the data stream an index using a variable length code; and

perform a selection using the index.

26. The apparatus of claim 14 , wherein the instructions, when executed by the processor, further cause the processor to:

determine a ranking of a set of intra prediction modes; and, subsequently, resample the second template.

27. A method for block-wise decoding a picture from a data stream, the method supporting at least one intra-prediction mode according to which an intra-prediction signal for a block of a predetermined size of the picture is determined by applying a first template of samples which neighbors the block, the method comprising:

resampling a second template of samples neighboring a current block, so as to have a dimension of the first template so as to acquire a resampled template, wherein the resampling comprises downsampling the second template to acquire the resampled template,

performing an intra prediction by applying the resampled template of samples so as to acquire a preliminary intra-prediction signal, and

resampling the preliminary intra-prediction signal so as to have a dimension of the current block so as to acquire the intra-prediction signal for the current block, wherein resampling the preliminary intra-prediction signal comprises upsampling the preliminary intra-prediction signal.

28. The method of claim 27 ,

the method supporting at least one intra-prediction mode according to which the intra-prediction signal for the block of the predetermined size of the picture is determined by applying the first template of samples which neighbors the block onto a neural network,

wherein performing the intra prediction comprises applying the resampled template of samples onto the neural network so as to acquire the preliminary intra-prediction signal.

29. The method of claim 27 , comprising:

transforming the preliminary intra-prediction signal from a spatial domain into a transform domain; and

resampling the preliminary intra-prediction signal in the transform domain.

30. The method of claim 27 , comprising:

resampling the preliminary intra-prediction signal in a spatial domain by performing a bilinear interpolation.

31. The method of claim 27 , comprising:

determining a ranking of a set of intra prediction modes; and, subsequently, resampling the second template.

32. A method for block-wise encoding a picture into a data stream, the method supporting at least one intra-prediction mode according to which an intra-prediction signal for a block of a predetermined size of the picture is determined by applying a first template of samples which neighbors the block, the method comprising:

resampling a second template of samples neighboring a current block, so as to have a dimension of the first template so as to acquire a resampled template, wherein the resampling comprises downsampling the second template to acquire the resampled template,

performing an intra prediction by applying the resampled template of samples so as to acquire a preliminary intra-prediction signal, and

resampling the preliminary intra-prediction signal so as to have a dimension of the current block so as to acquire the intra-prediction signal for the current block, wherein resampling the preliminary intra-prediction signal comprises upsampling the preliminary intra-prediction signal.

33. The method of claim 32 , the method supporting at least one intra-prediction mode according to which the intra-prediction signal for the block of the predetermined size of the picture is determined by applying the first template of samples which neighbors the block onto a neural network,

wherein performing the intra prediction comprises applying the resampled template of samples onto the neural network so as to acquire the preliminary intra-prediction signal.

34. The method of claim 32 , comprising:

transforming the preliminary intra-prediction signal from a spatial domain into a transform domain; and

resampling the preliminary intra-prediction signal in the transform domain.

35. The method of claim 32 , comprising:

resampling the preliminary intra-prediction signal in the spatial domain by performing a bilinear interpolation.

36. The method of claim 32 , comprising:

determining a ranking of a set of intra prediction modes; and, subsequently, resampling the second template.

37. A non-transitory digital storage medium having a computer program stored thereon to perform a method for block-wise decoding a picture from a data stream, the method supporting at least one intra-prediction mode according to which an intra-prediction signal for a block of a predetermined size of the picture is determined by applying a first template of samples which neighbors the block, the method comprising:

resampling a second template of samples neighboring a current block, so as to have a dimension of the first template so as to acquire a resampled template, wherein the resampling comprises downsampling the second template to acquire the resampled template,

performing an intra prediction by applying the resampled template of samples so as to acquire a preliminary intra-prediction signal, and

resampling the preliminary intra-prediction signal so as to have a dimension of the current block so as to acquire the intra-prediction signal for the current block, wherein resampling the preliminary intra-prediction signal comprises upsampling the preliminary intra-prediction signal,

when said computer program is run by a computer.

38. A non-transitory digital storage medium having a computer program stored thereon to perform a method for block-wise encoding a picture into a data stream, the method supporting at least one intra-prediction mode according to which an intra-prediction signal for a block of a predetermined size of the picture is determined by applying a first template of samples which neighbors the block, the method comprising:

resampling a second template of samples neighboring a current block, so as to have a dimension of the first template so as to acquire a resampled template, wherein the resampling comprises downsampling the second template to acquire the resampled template,

performing an intra prediction by applying the resampled template of samples so as to acquire a preliminary intra-prediction signal, and

resampling the preliminary intra-prediction signal so as to have a dimension of the current block so as to acquire the intra-prediction signal for the current block, wherein resampling the preliminary intra-prediction signal comprises upsampling the preliminary intra-prediction signal,

when said computer program is run by a computer.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2021
From: PFAFF, JONATHAN; HELLE, PHILIPP; MERKLE, PHILIPP; STALLENBERGER, BJÖRN; SIEKMANN, MISCHA; WINKEN, MARTIN; WIECKOWSKI, ADAM; SAMEK, WOJCIECH; KALTENSTADLER, STEPHAN; SCHWARZ, HEIKO; MARPE, DETLEV; WIEGAND, THOMAS
To: FRAUNHOFER-GESELLSCHAFT ZUR FÖRDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
Reel/Frame 057318/0735 →
Priority Claims (1)
EP 18165224 · Mar 29, 2018 · regional
Continuity (2)
Continuation PCTEP2019057882 · Mar 28, 2019
Related Publication 20210014531A1 · Jan 14, 2021