Systems, methods, and computer program products for scalable video coding based on coefficient sampling
Methods, systems, and computer program products for the generation of multiple layers of scaled encoded video data compatible with the HEVC standard. Residue from prediction processing may be transformed into coefficients in the frequency domain. The coefficients may then be sampled to create a layer of encoded data. The coefficients may be sampled in different ways to create multiple respective layers. The layers may then be multiplexed and sent to a decoder. There, one or more of the layers may be chosen. The choice of certain layer(s) may be dependent on the desired attributes of the resulting video. A certain level of video quality, frame rate, resolution, and/or bit depth may be desired, for example. The coefficients in the chosen layers may then be assembled to create a version of the residue to be used in video decoding.
1. A method of scalable video coding (SVC), comprising:
receiving a residue from a video compression process;
generating a coefficient matrix by performing a frequency transformation of the residue;
sampling the coefficient matrix according to two or more predefined sampling patterns, each of the sampling patterns comprising a sampling period and a sampling phase, wherein the phase specifies one or more coefficients within the sampling period that are to be included in a sampled subset of the coefficients;
quantizing and entropy encoding a first sampled subset of the coefficients as a first layer of encoded video data;
quantizing and entropy encoding a second sampled subset of the coefficients as a second layer of encoded video data; and
multiplexing the layers of encoded video data into a bitstream.
2. The method of claim 1 , wherein:
the sampling of the coefficient matrix begins at a start position, and ends at an end position within the matrix;
a first of the sampling patterns has a first sampling phase; and
a second of the sampling patterns has a second sampling phase, different from the first sampling phase.
3. The method of claim 2 , wherein a first and a second of the sampling patterns separate the coefficients into consecutive, non-overlapping groups, and the first and second sampling patterns sample different coefficient positions from consecutive ones of the groups until reaching the end position.
4. The method of claim 1 , wherein the sampling parameters are adaptively determined based on a quantization parameter (QP) value.
5. The method of claim 1 , wherein the sampling parameters are generated at an encoder
based on characteristics of a current picture and/or previously encoded pictures.
6. The method of claim 1 , wherein the sampling parameters are generated at the decoder
based on characteristics of previously coded neighboring pictures.
7. The method of claim 1 , wherein the sampling parameters are based on a transform
core used in the transformation.
8. The method of claim 1 , wherein different channels use different sampling parameters.
9. The method of claim 1 , further comprising scanning the coefficients into a one-dimensional
form before the sampling of the coefficient matrix.
10. The method of claim 1 , wherein the coefficient matrix is two-dimensional, and wherein the predefined sampling period comprises a horizontal sampling period and vertical sampling period.
11. The method of claim 1 ,
wherein the first and second layers are both enhancement layers; and wherein the method further comprises:
generating an unsampled coefficient group associated with frequencies below those corresponding to the sample range; and
quantizing and entropy encoding the unsampled coefficient group as a base layer of encoded video data.
12. A method of scalable video coding (SVC), comprising:
receiving a multiplexed bitstream comprising multiple layers of encoded video data;
decoding the multiplexed bitstream, wherein the decoding includes selecting from the multiplexed bitstream a subset of the layers;
deriving from the layers a sampled subset of coefficients that are within a sample range and satisfy different predetermined sampling patterns comprising a sampling period and a sampling phase, wherein the phase specifies one or more coefficients within the sampling period that are included in each of the sampled subsets of the coefficients;
assembling the sampled subsets of coefficients into a coefficient matrix that represents, at least in part, residue of an SVC compression process; and
performing an inverse transform on the coefficient matrix, to create a version of the residue.
13. The method of claim 12 , further comprising deriving from the bitstream an unsampled coefficient group associated with frequencies below those corresponding to the sample range.
14. The method of claim 12 , wherein said deriving of the sample of coefficients from each of the layers comprises entropy decoding the layer, and inverse quantizing the entropy decoded layer.
15. The method of claim 12 , wherein selecting from the multiplexed bitstream a subset of the layers comprising selecting a subset of the plurality of layers to achieve a desired video quality.
16. A scalable video decoder, comprising:
an electronic memory to store a multiplexed bitstream comprising base layer (BL) video data and enhancement layer (EL) video data; and
one or more microprocessors coupled to the electronic memory, the one or more microprocessors to perform the method of claim 12 .
17. The decoder of claim 16 , wherein:
the sampling of the coefficient matrix begins at a start position, and ends at an end position within the matrix;
a first of the sampling patterns has a first sampling phase; and
a second of the sampling patterns has a second sampling phase, different from the first sampling phase.
18. A scalable video encoder, comprising:
an electronic memory to store a residue from a predictive video compression process; and
one or more microprocessors communicatively coupled to the memory, the one or more microprocessors to:
receive a residue from a video compression process;
generate a coefficient matrix by performing a frequency transformation of the residue;
sample the coefficient matrix according to two or more predefined sampling patterns, each of the sampling patterns comprising a sampling period and a sampling phase, wherein the phase specifies one or more coefficients within the sampling period that are to be included in a sampled subset of the coefficients;
quantize and entropy encode a first sampled subset of the coefficients as a first layer of encoded video data;
quantize and entropy encode a second sampled subset of the coefficients as a second layer of encoded video data; and
multiplex the layers of encoded video data into a bitstream.
19. The encoder of claim 18 , wherein:
the sampling of the coefficient matrix begins at a start position, and ends at an end position within the matrix;
a first of the sampling patterns has a first sampling phase; and
a second of the sampling patterns has a second sampling phase, different from the first sampling phase.
20. The encoder of claim 19 , wherein a first and a second of the sampling patterns separate the coefficients into consecutive, non-overlapping groups, and the first and second sampling patterns sample different coefficient positions from consecutive ones of the groups until reaching the end position.