Enhancement layer residual prediction for bit depth scalability using hierarchical LUTs
A scalable video bitstream may have an H.264/AVC compatible base layer and a scalable enhancement layer, where scalability refers to color bit-depth. According to the invention, BL information is bit-depth upsampled using separate look-up tables for inverse tone mapping on two or more hierarchy levels, such as picture level, slice level or MB level. The look-up tables are differentially encoded and included in header information. Bit-depth upsampling is a process that increases the number of values that each pixel can have, corresponding to the pixels color intensity. The upsampled base layer data are used to predict the collocated enhancement layer, based on said look-up tables. The upsampling is done at the encoder side and in the same manner at the decoder side, wherein the upsampling may refer to temporal, spatial and bit depth characteristics. Thus, the bit-depth upsampling is compatible with texture upsampling.
1. A method for encoding video data having a base layer and an enhancement layer, wherein pixels of the base layer have less color bit depth than pixels of the enhancement layer, the method comprising:
encoding base layer data on a first granularity level, wherein the base layer data are intra-coded;
reconstructing the encoded base layer data;
generating for the intra-coded base layer data a first tone mapping table defining a tone mapping between original enhancement layer data and corresponding reconstructed base layer data;
generating for a fraction of the intra-coded base layer data a different second tone mapping table defining a tone mapping between said fraction of the original enhancement layer data and the corresponding fraction of the reconstructed base layer data;
generating a difference table representing the difference between the first tone mapping table and the second tone mapping table;
bit-depth upsampling the reconstructed base layer data based on said first and second tone mapping tables, wherein the second tone mapping table is used only for said fraction of the reconstructed base layer and the first tone mapping table is used for the remainder of the reconstructed base layer, and wherein a predicted version of the corresponding enhancement layer data is obtained that has higher bit-depth resolution than the base layer data;
generating an enhancement layer residual being the difference between the original enhancement layer data and the corresponding predicted version of the enhancement layer data; and
encoding the enhancement layer residual, the first tone mapping table and said difference table, wherein the encoded first tone mapping table is associated with the encoded base layer or enhancement layer data and the difference table is associated with said fraction of the encoded base layer data or enhancement layer data.
2. The method according to claim 1 , further comprising upsampling the reconstructed base layer data before said bit-depth upsampling, wherein a first predicted version of the corresponding enhancement layer data is obtained that has higher spatial, temporal or SNR resolution than the base layer data and that is used for said bit-depth upsampling.
3. The method according to claim 1 , wherein the first tone mapping table or the difference table, or both, are entropy coded, wherein an entry is represented by a differential value relating to a previous entry.
4. The method according to claim 1 , wherein separate tone mapping tables are generated for luminance and chrominance channels.
5. The method according to claim 1 , wherein no tone mapping tables are generated for inter coded base layer data.
6. A method for decoding video data having a base layer and an enhancement layer, the method comprising:
extracting from encoded enhancement layer data or base layer data first and second tone mapping data relating to intra-coded enhancement layer data;
reconstructing a first tone mapping table from the extracted tone mapping data;
reconstructing a second tone mapping table from the extracted tone mapping data and said reconstructed first tone mapping table, wherein the extracted tone mapping data represent the difference between said first tone mapping table and said second tone mapping table;
determining a first encoding unit to which the first tone mapping table relates, and a second encoding unit to which the second tone mapping table relates, wherein the second encoding unit is a fraction of said first encoding unit;
performing inverse quantization and inverse transformation on received base layer data and enhancement layer data, wherein inverse quantized and inverse transformed enhancement layer data comprise a residual;
reconstructing intra-coded base layer data;
upsampling the reconstructed base layer data, wherein a value depth per pixel is increased and wherein for pixels in said second encoding unit the second tone mapping table is used and for remaining pixels of the first encoding unit the first tone mapping table is used, and wherein predicted enhancement layer data are obtained; and
reconstructing from the predicted enhancement layer data and the inverse quantized and inverse transformed enhancement layer data reconstructed enhancement layer video data.
7. The method according to claim 6 , wherein upsampling comprises also one or more of spatial, temporal or SNR upsampling.
8. The method according to claim 6 , wherein said first or said differential second tone mapping table, or both, are differentially encoded.
9. An apparatus for encoding video data having a base layer and an enhancement layer, wherein pixels of the base layer have less color bit depth than pixels of the enhancement layer, the apparatus comprising a memory device and a processor configured to:
encode base layer data on a first granularity level, wherein the base layer data are intra-coded;
reconstruct the encoded base layer data;
generate an enhancement layer residual; and
encode the enhancement layer residual;
and the apparatus further comprising:
a look-up table (LUT) generation block for generating for the intra-coded base layer data a first tone mapping table defining a tone mapping between original enhancement layer data and corresponding reconstructed base layer data,
and for generating for a fraction of the intra-coded base layer data a different second tone mapping table defining a tone mapping between said fraction of the original enhancement layer data and the corresponding fraction of the reconstructed base layer data,
and for generating a difference table representing the difference between the first tone mapping table and the second tone mapping table;
a bit-depth upsampling block for bit-depth upsampling the reconstructed base layer data based on said first and second tone mapping tables, wherein the second tone mapping table is used only for said fraction of the reconstructed base layer and the first tone mapping table is used for the remainder of the reconstructed base layer, and wherein a predicted version of the corresponding enhancement layer data is obtained that has higher bit-depth resolution than the base layer data;
a difference generator for generating said enhancement layer residual being the difference between the original enhancement layer data and the corresponding predicted version of the enhancement layer data; and
a LUT entropy coding unit for encoding the first tone mapping table and said difference table, wherein the encoded first tone mapping table is associated with the encoded base layer or enhancement layer data and the difference table is associated with said fraction of the encoded base layer data or enhancement layer data.
10. An apparatus for decoding video data having a base layer and an enhancement layer, comprising a memory device and a processor configured to:
extract from encoded enhancement layer data or base layer data first and second tone mapping data relating to intra-coded enhancement layer data;
reconstruct in a look-up table (LUT) decoding unit a first tone mapping table from the extracted tone mapping data,
reconstruct in the LUT decoding unit a second tone mapping table from the extracted tone mapping data and said reconstructed first tone mapping table, wherein the extracted tone mapping data represent the difference between said first tone mapping table and said second tone mapping table;
determine a first encoding unit to which the first tone mapping table relates, and a second encoding unit to which the second tone mapping table relates, wherein the second encoding unit is a fraction of said first encoding unit;
perform inverse quantization and inverse transformation on received base layer data and enhancement layer data, wherein inverse quantized and inverse transformed enhancement layer data comprise a residual;
reconstruct intra-coded base layer data;
upsample in a bit-depth prediction unit reconstructed base layer data, wherein a value depth per pixel is increased, and wherein for pixels in said second encoding unit the second tone mapping table is used and for remaining pixels of the first encoding unit the first tone mapping table is used, and wherein predicted enhancement layer data are obtained; and
reconstruct from the predicted enhancement layer data and the inverse quantized and inverse transformed enhancement layer data reconstructed enhancement layer video data.
11. The apparatus according to claim 10 , wherein the first and second tone mapping data are extracted from one or more headers of intra-coded enhancement layer data.
12. The method according to claim 1 , wherein separate tone mapping tables are generated for each chrominance channel.
13. The method according to claim 6 , wherein separate tone mapping tables are generated for each chrominance channel.
14. The apparatus according to claim 9 , wherein separate tone mapping tables are generated for each chrominance channel.
15. The apparatus according to claim 10 , wherein separate tone mapping tables are generated for each chrominance channel.
16. A method for encoding a video image having a plurality of slices, each slice having a base layer and an enhancement layer, wherein pixels of the base layer have less color bit depth than pixels of the enhancement layer, the method comprising:
encoding intra-coded base layer data of the video image on a first granularity level;
reconstructing the encoded base layer data;
generating a first tone mapping table defining a tone mapping between original enhancement layer data and corresponding reconstructed base layer data;
generating at least one different second tone mapping table defining a tone mapping between a slice of the original enhancement layer data of the image and the corresponding slice of the reconstructed base layer data;
generating a difference table representing the difference between the first tone mapping table and the second tone mapping table;
bit-depth upsampling the reconstructed base layer data based on said first and second tone mapping tables, wherein the second tone mapping table is used only for said slice of the reconstructed base layer and the first tone mapping table is used for the remaining slice or slices of the reconstructed base layer, and wherein a predicted version of the corresponding enhancement layer data is obtained that has higher bit-depth resolution than the base layer data;
generating an enhancement layer residual being the difference between the original enhancement layer data and the corresponding predicted version of the enhancement layer data; and
encoding the enhancement layer residual, the first tone mapping table and said difference table, wherein the encoded first tone mapping table is associated with the encoded base layer or enhancement layer data and the difference table is associated with said slice of the encoded base layer data or enhancement layer data.
17. A method for decoding a video image having a plurality of slices, each slice having a base layer and an enhancement layer, the method comprising:
extracting from encoded enhancement layer data or base layer data first and second tone mapping data relating to intra-coded enhancement layer data;
reconstructing a first tone mapping table from the extracted tone mapping data;
reconstructing a second tone mapping table from the extracted tone mapping data and said reconstructed first tone mapping table, wherein the extracted tone mapping data represent the difference between said first and second tone mapping table;
determining, among the plurality of slices of said video image, at least one first slice to which the first tone mapping table relates, and a second slice to which the second tone mapping table relates;
performing inverse quantization and inverse transformation on received base layer data and enhancement layer data, wherein inverse quantized and inverse transformed enhancement layer data comprise a residual;
reconstructing intra-coded base layer data;
upsampling the reconstructed base layer data, wherein a value depth per pixel is increased, and wherein for pixels in said second slice the second tone mapping table is used and for pixels of the at least one first slice the first tone mapping table is used, and wherein predicted enhancement layer data for each slice are obtained; and
reconstructing from the predicted enhancement layer data and the inverse quantized and inverse transformed enhancement layer data reconstructed enhancement layer video data for each slice of the video image.