Efficient scalable coding concept
Scalable coding concepts are described. One aspect improves parallel decoding of inter-dependent layers of a multi-layer video data stream by introducing a long-term syntax element structure for guaranteeing that during a predetermined time period the pictures of the dependent layer are subdivided so that borders of the spatial segments of the pictures of the second layer and the spatial segments of the first layer overlay. Another aspect concerns upsampling from base layer to enhancement layer. Another aspect introduces a long-term syntax element structure allowing the decoder to determine the inter-layer offset for a predetermined time period. Another aspect introduces a type indicator field changing a way a layer indicator field within the NAL unit headers is to be interpreted. Another aspect allows different codecs/standards to be used for the different layers. Another aspect concerns a syntax element structure which indicates the inter-layer offset in units of the base layer blocks.
1. A video decoder for decoding a multi-layer video data stream into which a video is coded in layers using inter-layer prediction from a first layer to a second layer, the video decoder comprising a processor configured to:
decode a long-term syntax element of the multi-layer video data stream, wherein the long-term syntax element indicates whether one or more spatial regions in pictures of the first layer are not used for inter-layer prediction for decoding of a picture of the second layer in a predetermined time period;
based on the decoding of the long-term syntax element indicating that the one or more spatial regions in pictures of the first layer are not used for inter-layer prediction for decoding of the picture of the second layer,
periodically determine, in time intervals smaller than the predetermined time period, a number of the one or more spatial regions in each picture of the first layer that are not used for inter-layer prediction for decoding of the picture of the second layer based on first short-term syntax elements of the multi-layer video data stream; and
periodically determine, in the time intervals smaller than the predetermined time period, sizes and locations of the spatial regions of the pictures of the first layer and spatial regions of pictures of the second layer, and spatial sampling resolutions of the pictures of the first layer and the pictures of the second layer, respectively, based on second short-term syntax elements of the multi-layer video data stream.
2. The video decoder of claim 1 , wherein the processor is configured to parallel decode the multi-layer video data stream in spatial segments into which pictures of the layers are partitioned by sequentially traversing the spatial segments in a temporally overlapping manner with an inter-layer delay between traversal of spatial segments of pictures of the first layer relative to traversal of spatial segments of pictures of the second layer.
3. The video decoder of claim 1 , wherein the processor is configured to, in parallel, decode the multi-layer video data stream in sub-streams out of which pictures of the layers are partitioned and which are composed of rows of blocks into which the pictures of the layers are sub-divided, using wavefront parallel processing by sequentially traversing the substreams in a temporally overlapping manner with an intra-picture inter-substream delay between traversal of immediately consecutive substreams of the same picture and an inter-layer offset between traversal of substreams of pictures of the first layer relative to traversal of substreams of pictures of the second layer.
4. The video decoder of claim 3 , wherein the processor is configured to decode the sub-streams in parallel with supporting intra-picture spatial prediction crossing boundaries of the sub-streams.
5. The video decoder of claim 1 , wherein the processor is configured to decode the multi-layer video data stream in tiles out of which pictures of the layers are partitioned, with traversing the tiles at a tile order among the tiles within each of the pictures of the first and second layers, and decoding immediately succeeding tiles of pictures of the first layer and immediately succeeding tiles of pictures of the second layer in parallel with an inter-layer offset between the traversal of the tiles of the pictures of the first layer relative to the traversal of tiles of the pictures of the second layer.
6. The video decoder of claim 5 , wherein the processor is configured to decode the pictures of the first and second layers using intra-picture spatial prediction with disrupting the intra-picture spatial prediction for each tile at a boundary of the respective tile.
7. The video decoder of claim 5 , wherein the processor is configured to use a value of the long-term syntax element in determining the inter-layer offset by using the value of the long-term syntax element as a measure for the inter-layer offset in units of spatial segments of the pictures of the first layer.
8. The video decoder of claim 5 , wherein the processor is configured to use a value of the long-term syntax element in determining the inter-layer offset by using the value of the long-term syntax element as a number of spatial segments of the pictures of the first layer by which the decoding of a first spatial segment of the pictures of the second layer is to be delayed relative to commencing the decoding and traversal of the pictures of the first layer.
9. A video encoder for encoding a video into layers of a multi-layer video data stream using inter-layer prediction from a first layer to a second layer, the video encoder comprising a processor configured to:
encode a long-term syntax element into the multi-layer video data stream, wherein the long-term syntax element indicates whether one or more spatial regions in pictures of the first layer are not used for inter-layer prediction for coding of a picture of the second layer in a predetermined time period;
based on the long-term syntax element indicating that the one or more spatial regions in pictures of the first layer are not used for inter-layer prediction for coding of the picture of the second layer,
encoding first short-term syntax elements into the multi-layer video data stream, wherein the first short-term syntax elements are used to periodically determine, in time intervals smaller than the predetermined time period, a number of the one or more spatial regions in each picture of the first layer that are not used for inter-layer prediction for coding of the picture of the second layer; and
encoding second short-term syntax elements into the multi-layer video data stream, wherein the second short-term syntax elements are used to periodically determine, in the time intervals smaller than the predetermined time period, sizes and locations of the spatial regions of the pictures of the first layer and spatial regions of pictures of the second layer, and spatial sampling resolutions of the pictures of the first layer and the pictures of the second layer, respectively.
10. The video encoder of claim 9 , wherein the processor is configured to parallel encode the video into the multi-layer video data stream in spatial segments into which pictures of the layers are partitioned by sequentially traversing the spatial segments in a temporally overlapping manner with an inter-layer delay between traversal of spatial segments of pictures of the first layer relative to traversal of spatial segments of pictures of the second layer.
11. The video encoder of claim 9 , wherein the processor is configured to, in parallel, encode the video into the multi-layer video data stream in sub-streams out of which pictures of the layers are partitioned and which are composed of rows of blocks into which the pictures of the layers are sub-divided, using wavefront parallel processing by sequentially traversing the substreams in a temporally overlapping manner with an intra-picture inter-substream delay between traversal of immediately consecutive substreams of the same picture and an inter-layer offset between traversal of substreams of pictures of the first layer relative to traversal of substreams of pictures of the second layer.
12. The video encoder of claim 11 , wherein the processor is configured to encode the sub-streams in parallel with supporting intra-picture spatial prediction crossing boundaries of the sub-streams.
13. The video encoder of claim 9 , wherein the processor is configured to encode the video into the multi-layer video data stream in tiles out of which pictures of the layers are partitioned, with traversing the tiles at a tile order among the tiles within each of the pictures of the first and second layers, and encoding immediately succeeding tiles of pictures of the first layer and immediately succeeding tiles of pictures of the second layer in parallel with an inter-layer offset between the traversal of the tiles of the pictures of the first layer relative to the traversal of tiles of the pictures of the second layer.
14. The video encoder of claim 13 , wherein the processor is configured to encode the pictures of the first and second layers using intra-picture spatial prediction with disrupting the intra-picture spatial prediction for each tile at a boundary of the respective tile.