IP Library Granted Patent US 9,843,800
Granted Patent B2
US 9,843,800 · App. 14/324,425 · Granted Dec 12, 2017

Quality scalable coding with mapping different ranges of bit depths

Inventors: Detlev Marpe (Berlin, DE); Martin Winken (Berlin, DE); Heiko Schwarz (Berlin, DE); Thomas Wiegand (Berlin, DE)
Assignee: GE VIDEO COMPRESSION, LLC
H04N19/00569H04N19/117H04N19/184H04N19/186H04N19/187H04N19/30H04N19/36H04N19/50H04N19/61
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,843,800
App. No.
14/324,425
Granted
Dec 12, 2017
Kind
B2
Abstract

A more efficient way of addressing different bit-depths, or different bit-depths and chroma sampling format requirements is achieved by using a low bit-depth and/or low-chroma resolution representation for providing a respective base layer data stream representing this low bit-depth and/or low-chroma resolution representation as well as for providing a higher bit-depth and/or higher chroma resolution representation so that a respective prediction residual may be encoded in order to obtain a higher bit-depth and/or higher chroma resolution representation. By this measure, an encoder is enabled to store a base-quality representation of a picture or a video sequence, which can be decoded by any legacy decoder or video decoder, together with an enhancement signal for higher bit-depth and/or reduced chroma sub-sampling, which may be ignored by legacy decoders or video decoders.

Claims (63)

1. An encoder for encoding a picture into a quality-scalable data stream, comprising:

a base encoder for encoding the picture into a base encoded data stream representing a representation of the picture with a first picture sample bit depth;

a predictor for providing a prediction of the picture based on the representation of the picture with the first picture sample bit depth, the prediction of the picture comprising a second picture sample bit depth that is higher than the first picture sample bit depth, the predictor comprising a mapper for performing, on the representation of the picture with the first picture sample bit depth, a mapping of samples of the picture from a first dynamic range corresponding to the first picture sample bit depth to a second dynamic range greater than the first dynamic range and corresponding to the second picture sample bit depth to obtain the prediction of the picture,

wherein the mapping comprises interpolation of samples corresponding to the second picture sample bit depth based on the samples corresponding to the first picture sample bit depth and a bit-shift operation;

a residual encoder for encoding a prediction residual of the prediction into a bit-depth enhancement layer data stream; and

a combiner for forming the quality-scalable data stream based at least on the base encoded data stream, and the bit-depth enhancement layer data stream.

2. The encoder according to claim 1 , wherein the predictor comprises a reconstructor for reconstructing a low bit-depth reconstruction picture as the representation of the picture with the first picture sample bit depth based on the base encoded data stream, the low bit-depth reconstruction picture comprising the first picture sample bit depth.

3. The encoder according to claim 1 , wherein the mapper is adapted to linearly map the samples from the first dynamic range to the second dynamic range.

4. The encoder according to claim 1 , wherein the mapper is adapted to piece-wise linearly map the samples from the first dynamic range to the second dynamic range and forward interpolation point information defining the piece-wise linear mapping to the combiner for formation of the quality-scalable data stream also based on the interpolation point information.

5. The encoder according to claim 1 , wherein the mapper is adapted to perform the mapping by using the samples as an index to a look-up table.

6. The encoder according to claim 1 , wherein the base encoder comprises

a mapper for mapping samples representing the picture with the second picture sample bit depth from the second dynamic range to the first dynamic range corresponding to the first picture sample bit depth to acquire a quality-reduced picture; and

an encoder for encoding the quality-reduced picture to acquire the base encoded data stream.

7. The encoder according to claim 1 , wherein the picture is part of a video signal.

8. The encoder according to claim 7 , wherein the base encoder comprises a predictor for performing a temporal prediction based on pictures reconstructed from the base encoded data stream, the predictor for performing a temporal prediction concurrently forming a reconstructor for reconstructing a low bit-depth reconstruction picture based on the base encoded data stream for the predictor, the low bit-depth reconstruction picture comprising the first picture sample bit depth, wherein the predictor further comprises a mapper for performing, on the low bit-depth reconstruction picture, a mapping of samples from a first dynamic range corresponding to the first picture sample bit depth to a second dynamic range greater than the first dynamic range and corresponding to the second picture sample bit depth to acquire the prediction of the picture.

9. The encoder according to claim 1 , wherein the representation of the picture comprises a luma resolution and a first chroma resolution, and the predictor further comprises

an applicator for applying a spatial interpolation filter to acquire the prediction of the picture from the representation of the picture with the luma resolution and the first chroma resolution, the prediction of the picture comprising the luma resolution and a second chroma resolution being higher than the first chroma resolution.

10. The encoder according to claim 9 , wherein the spatial interpolation filter is selected among a group comprising a linear interpolation filter, a bilinear interpolation filter, and a FIR interpolation filter.

11. The encoder according to claim 9 , wherein the base encoder is adapted such that the base encoded data stream represents the picture in a first color space domain, and wherein the applicator is adapted to apply the spatial interpolation filter, and then perform a color space transformation from the first color space domain to a second color space domain to acquire the prediction of the picture.

12. The encoder according to claim 9 , wherein the applicator is adapted to forward filter information relating to the spatial interpolation filter to the combiner for formation of the quality-scalable data stream also based on the filter information.

13. The encoder according to claim 6 wherein the base encoder comprises

a receiver for receiving the picture with the luma resolution and the second chroma resolution and reducing the second chroma resolution to the first chroma resolution to acquire a quality-reduced picture; and

a sub-encoder for encoding the quality-reduced picture to acquire the base encoded data stream.

14. The encoder according to claim 1 , wherein the predictor is adapted to perform a dithering in order to acquire the prediction of the picture.

15. A decoder for decoding a quality-scalable data stream into which a picture is encoded,

the quality-scalable data stream comprising a base layer data stream representing the picture with a first picture sample bit depth and a bit-depth enhancement layer data stream representing a prediction residual with a second picture sample bit depth that is higher than the first picture sample bit depth,

the decoder comprising:

a first sub-decoder for decoding the base layer data stream into a lower bit-depth reconstructed picture;

a second sub-decoder for decoding the bit-depth enhancement data stream into the prediction residual;

a provider for providing a prediction of the picture based on the lower bit-depth reconstructed picture, the prediction of the picture comprising the second picture sample bit depth, wherein the provider is adapted to perform, on the lower bit-depth reconstructed picture, a mapping of samples of the picture from a first dynamic range corresponding to the first picture sample bit depth to a second dynamic range greater than the first dynamic range and corresponding to the second picture sample bit depth to obtain the prediction of the picture, wherein the mapping comprises interpolation of samples corresponding to the second picture sample bit depth based on the samples corresponding to the first picture sample bit depth and a bit-shift operation; and

a reconstructor for reconstructing the picture with the second picture sample bit depth based on the prediction and the prediction residual.

16. The decoder according to claim 15 , wherein the provider is adapted to, according to a first or second mathematical mapping formula, linearly map the samples from the first dynamic range to the second dynamic range.

17. The decoder according to claim 15 , wherein the provider is adapted to, according to a first or second mathematical mapping formula, piece-wise linearly map the samples from the first dynamic range to the second dynamic range by use of interpolation point information retrieved from the quality-scalable data stream.

18. The decoder according to claim 15 , wherein the provider is adapted to, according to a first or second mathematical mapping formula, perform the mapping by use of the samples as an index to a look-up table.

19. The decoder according to claim 15 , wherein the picture is part of a video signal.

20. The decoder according to claim 15 , wherein the lower bit-depth reconstructed picture comprises a luma resolution and a first chroma resolution, and the provider is adapted to apply a spatial interpolation filter to acquire the prediction of the picture from the lower bit-depth reconstructed picture, the prediction of the picture comprising the luma resolution and a second chroma resolution being higher than the first chroma resolution.

21. The decoder according to claim 20 , wherein the spatial interpolation filter is selected among the group comprising a linear interpolation filter, a bilinear interpolation filter, and a FIR interpolation filter.

22. The decoder according to claim 20 , wherein the provider is adapted to apply the spatial interpolation filter and then perform a color space transformation from a first color space domain to a second color space domain to acquire the prediction of the picture.

23. The decoder according to claim 20 , wherein the provider is adapted to perform the application based on filter information retrieved from the quality-scalable data stream.

24. The decoder according to claim 15 , wherein the predictor is adapted to perform a dithering in order to acquire the prediction of the picture.

25. A method for encoding a picture into a quality-scalable data stream, comprising:

encoding the picture into a base encoded data stream representing a representation of the picture with a first picture sample bit depth;

providing a prediction of the picture based on the representation of the picture with the first picture sample bit depth, the prediction of the picture comprising a second picture sample bit depth that is higher than the first picture sample bit depth, the provision comprising performing, on the representation of the picture with the first picture sample bit depth, a mapping of samples of the picture from a first dynamic range corresponding to the first picture sample bit depth to a second dynamic range greater than the first dynamic range and corresponding to the second picture sample bit depth to obtain the prediction of the picture, wherein the mapping comprises interpolation of samples corresponding to the second picture sample bit depth based on the samples corresponding to the first picture sample bit depth and a bit-shift operation;

encoding a prediction residual of the prediction into a bit-depth enhancement layer data stream; and

forming the quality-scalable data stream based at least on the base encoded data stream, and the bit-depth enhancement layer data stream.

26. A method for decoding a quality-scalable data stream into which a picture is encoded,

the quality-scalable data stream comprising a base layer data stream representing the picture with a first picture sample bit depth and a bit-depth enhancement layer data stream representing a prediction residual with a second picture sample bit depth that is higher than the first picture sample bit depth,

the method comprising:

decoding the base layer data stream into a lower bit-depth reconstructed picture;

decoding the bit-depth enhancement data stream into the prediction residual;

providing a prediction of the picture based on the lower bit-depth reconstructed picture, the prediction of the picture comprising the second picture sample bit depth, wherein the provision comprises performing, on the lower bit-depth reconstructed picture, a mapping of samples of the picture from a first dynamic range corresponding to the first picture sample bit depth to a second dynamic range greater than the first dynamic range and corresponding to the second picture sample bit depth to obtain the prediction of the picture, wherein the mapping comprises interpolation of samples corresponding to the second picture sample bit depth based on the samples corresponding to the first picture sample bit depth and a bit-shift operation; and

reconstructing the picture with the second picture sample bit depth based on the prediction and the prediction residual.

27. A non-transitory computer readable medium having a computer program comprising a program code for performing, when the computer program runs on a computer, a method for encoding a picture into a quality-scalable data stream, the method comprising:

encoding the picture into a base encoded data stream representing a representation of the picture with a first picture sample bit depth;

providing a prediction of the picture based on the representation of the picture with the first picture sample bit depth, the prediction of the picture comprising a second picture sample bit depth that is higher than the first picture sample bit depth, the provision comprising performing, on the representation of the picture with the first picture sample bit depth, a mapping of samples from a first dynamic range corresponding to the first picture sample bit depth to a second dynamic range greater than the first dynamic range and corresponding to the second picture sample bit depth to obtain the prediction of the picture, wherein the mapping comprises interpolation of samples corresponding to the second picture sample bit depth based on the samples corresponding to the first picture sample bit depth and a bit-shift;

encoding a prediction residual of the prediction into a bit-depth enhancement layer data stream; and

forming the quality-scalable data stream based at least on the base encoded data stream, and the bit-depth enhancement layer data stream.

28. A non-transitory computer readable medium having a computer program comprising a program code for performing, when the computer program runs on a computer, a method for decoding a quality-scalable data stream into which a picture is encoded, the quality-scalable data stream comprising a base layer data stream representing the picture with a first picture sample bit depth and a bit-depth enhancement layer data stream representing a prediction residual with a second picture sample bit depth that is higher than the first picture sample bit depth as well as mapping information, the method comprising:

decoding the base layer data stream into a lower bit-depth reconstructed picture;

decoding the bit-depth enhancement data stream into the prediction residual;

providing a prediction of the picture based on the lower bit-depth reconstructed picture, the prediction of the picture comprising the second picture sample bit depth, wherein the provision comprises performing, on the lower bit-depth reconstructed picture, a mapping of samples from a first dynamic range corresponding to the first picture sample bit depth to a second dynamic range greater than the first dynamic range and corresponding to the second picture sample bit depth to obtain the prediction of the picture, wherein the mapping comprises interpolation of samples corresponding to the second picture sample bit depth based on the samples corresponding to the first picture sample bit depth and a bit-shift operation; and

reconstructing the picture with the second picture sample bit depth based on the prediction and the prediction residual.

29. The encoder according to claim 1 , wherein the predictor further comprises an 8-tap FIR filter for interpolation.

Assignments (4)
CHANGE OF NAME Recorded Jan 30, 2026
From: GE VIDEO COMPRESSION, LLC
To: DOLBY VIDEO COMPRESSION, LLC
Reel/Frame 074536/0694 →
CHANGE OF NAME Recorded Nov 26, 2024
From: GE VIDEO COMPRESSION, LLC
To: DOLBY VIDEO COMPRESSION, LLC
Reel/Frame 069450/0113 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 9, 2015
From: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
To: GE VIDEO COMPRESSION, LLC
Reel/Frame 035806/0937 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 8, 2014
From: MARPE, DETLEV; WINKEN, MARTIN; SCHWARZ, HEIKO; WIEGAND, THOMAS
To: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V.
Reel/Frame 033259/0621 →
Continuity (2)
Continuation 12447005
Related Publication 20140321530A1 · Oct 30, 2014