Scalable video coding using inter-layer prediction contribution to enhancement layer prediction
A scalable video decoder is described which is configured to reconstruct a base layer signal from a coded data stream to obtain a reconstructed base layer signal; and reconstruct an enhancement layer signal including spatially or temporally predicting a portion of an enhancement layer signal, currently to be reconstructed, from an already reconstructed portion of the enhancement layer signal to obtain an enhancement layer internal prediction signal; forming, at the portion currently to be reconstructed, a weighted average of an inter-layer prediction signal obtained from the reconstructed base layer signal, and the enhancement layer internal prediction signal to obtain an enhancement layer prediction signal such that a weighting between the inter-layer prediction signal and the enhancement layer internal prediction signal varies over different spatial frequency components; and predictively reconstructing the enhancement layer signal using the enhancement layer prediction signal.
1. A video decoder comprising:
a first decoder configured for reconstructing, using a processor, a base layer signal from a coded data stream to obtain a reconstructed base layer signal; and
a second decoder configured for reconstructing, using a processor, an enhancement layer signal, the reconstructing comprising:
predicting, based on a reconstructed portion of the enhancement layer signal, a current portion of the enhancement layer signal to obtain an enhancement layer internal signal,
obtaining an inter-layer prediction signal associated with the current portion of the enhancement layer signal based on the reconstructed base layer signal,
extracting, from the coded data stream, weighting parameters for the inter-layer prediction signal and the enhancement layer internal signal,
determining a weighted average of the inter-layer prediction signal and the enhancement layer internal signal to obtain an enhancement layer prediction signal by: filtering the inter-layer prediction signal with a first filter to generate a filtered inter-layer prediction signal,
filtering the enhancement layer internal signal with a second filter to generate a filtered enhancement layer internal signal,
applying the weighting parameters to the filtered inter-layer prediction signal and the filtered enhancement layer internal signal, respectively, to obtain the enhancement layer prediction signal, and
predictively reconstructing the enhancement layer signal based on the enhancement layer prediction signal.
2. The video decoder according to claim 1 , wherein obtaining the inter-layer prediction signal comprises performing a resolution refinement to obtain the inter-layer prediction signal.
3. The video decoder according to claim 1 , wherein the weighting parameters for the inter-layer prediction signal and the enhancement layer internal signal vary over different spatial frequency components.
4. The video decoder according to claim 1 , wherein the first filter is a low-pass filter and the second filter is a high-pass filter.
5. The video decoder according to claim 1 , wherein the first and second filters form a quadrature mirror filter pair.
6. The video decoder according to claim 1 , wherein determining comprises the weighted average transforming the inter-layer prediction signal and the enhancement layer internal signal to obtain transform coefficients, and superimposing the transform coefficients using weighting parameters of different ratios therebetween for different spatial frequency components to obtain superimposed transform coefficients.
7. The video decoder according to claim 6 , wherein determining further comprises inverse transforming the superimposed transform coefficients to obtain the enhancement layer prediction signal.
8. The video decoder according to claim 6 , wherein predictively reconstructing the enhancement layer signal using the enhancement layer prediction signal comprises extracting transform coefficient levels for the enhancement layer signal from a coded data stream, performing a sum of the transform coefficient levels and the superimposed transform coefficients to obtain a transformed version of the enhancement layer signal, and performing an inverse transform of the transformed version of the enhancement layer signal to reconstruct the enhancement layer signal.
9. The video decoder according to claim 1 , wherein, for each spatial frequency component, a sum of the weighting parameters for the inter-layer prediction signal and the enhancement layer internal signal equal a value that is same for all spatial frequency components.
10. The video decoder according to claim 1 , wherein the weighting parameter for weighting the inter-layer prediction signal correspond to a low-pass transfer function, and the weighting parameter for weighting the enhancement layer internal signal correspond to a high-pass transfer function.
11. The video decoder according to claim 1 , wherein reconstructing the enhancement layer signal comprises
computing a difference signal between an already reconstructed portion of an enhancement layer signal and the inter-layer prediction signal,
spatially predicting the difference signal at a first portion collocated to the current portion of the enhancement layer signal from a second portion of the difference signal,
spatially neighboring the first portion and belonging to the already reconstructed portion of the enhancement layer signal to obtain a spatial intra prediction signal, and
combining the inter-layer prediction signal and the spatial intra prediction signal to reconstruct the enhancement layer prediction signal.
12. The video decoder according to claim 1 , wherein reconstructing the enhancement layer signal comprises
computing a difference signal between an already reconstructed portion of an enhancement layer signal and the inter-layer prediction signal,
temporally predicting the difference signal at a first portion collocated to the current portion of the enhancement layer signal from a second portion of the difference signal, belonging to previously reconstructed frames of the enhancement layer signal to obtain a temporally predicted difference signal, and
combining the inter-layer prediction signal and the temporally predicted difference signal to reconstruct the enhancement layer prediction signal.
13. A video encoder comprising:
a first coder configured encoding, using a processor, a base layer signal into a coded data stream in a manner allowing reconstructing a reconstructed base layer signal from the coded data stream; and
a second coder configured for encoding, using a processor, an enhancement layer signal, the encoding the enhancement layer signal comprising:
predicting a current portion of an enhancement layer from an encoded portion of the enhancement layer signal to obtain an enhancement layer internal signal,
obtaining an inter-layer prediction signal associated with the current portion of the enhancement layer signal based on the reconstructed base layer signal,
inserting, into the coded data stream, weighting parameters for the inter-layer prediction signal and the enhancement layer internal signal,
determining a weighted average of the inter-layer prediction signal and the enhancement layer internal signal to obtain an enhancement layer prediction signal by:
filtering the inter-layer prediction signal with a first filter to generate a filtered inter-layer prediction signal, and filtering the enhancement layer internal signal with a second filter to generate a filtered enhancement layer internal signal, and
applying the weighting parameters to the filtered inter-layer prediction signal and the filtered enhancement layer internal signal to obtain the enhancement layer prediction signal, and
predictively encoding the enhancement layer signal based on the enhancement layer prediction signal.
14. The video encoder according to claim 13 , wherein the weighting parameters for the inter-layer prediction signal and the enhancement layer internal signal vary over different spatial frequency components.
15. The video encoder according to claim 13 , wherein obtaining the inter-layer prediction signal comprises performing a resolution refinement to obtain the inter-layer prediction signal.
16. The video encoder according to claim 13 , wherein the weighting parameters for the inter-layer prediction signal and the enhancement layer internal signal vary over different spatial frequency components.
17. The video encoder according to claim 13 , wherein the first filter is a low-pass filter and the second filter is a high-pass filter.
18. The video encoder according to claim 13 , wherein the first and second filters form a quadrature mirror filter pair.
19. The video encoder according to claim 13 , wherein, for each spatial frequency component, a sum of the weighting parameters for the inter-layer prediction signal and the enhancement layer internal signal equal a value that is same for all spatial frequency components.
20. A non-transitory computer readable medium including a computer program comprising a program code for performing, when running on a computer, a method including:
reconstructing a base layer signal from a coded data stream to obtain a reconstructed base layer signal; and
reconstructing an enhancement layer signal, the reconstructing comprising:
predicting, based on a reconstructed portion of the enhancement layer signal, a current portion of the enhancement layer signal to obtain an enhancement layer internal signal,
obtaining an inter-layer prediction signal associated with the current portion of the enhancement layer signal based on the reconstructed base layer signal,
extracting, from the coded data stream, weighting parameters for the inter-layer prediction signal and the enhancement layer internal signal,
determining a weighted average of the inter-layer prediction signal and the enhancement layer internal signal to obtain an enhancement layer prediction signal by: filtering the inter-layer prediction signal with a first filter to generate a filtered inter-layer prediction signal, and filtering the enhancement layer internal signal with a second filter to generate a filtered inter-layer prediction signal, and applying the weighting parameters to the filtered inter-layer prediction signal and the filtered enhancement layer internal signal to obtain the enhancement layer prediction signal, and
predictively reconstructing the enhancement layer signal based on the enhancement layer prediction signal.