Cluster-based dependency signaling
The signalization of the inter-layer dependencies between layers of a multi-layered data stream is described. A good compromise between a too intensive restriction of the potential diversity of inter-layer dependencies on the one hand and a too complex signaling of the inter-layer dependencies on the other hand has been found by describing the inter-layer dependencies by way of a first inter-dependency syntax structure indicating inter-dependencies between pairs of different values representable by a base layer-ID and a second inter-dependency syntax structure indicating inter-dependencies between pairs of different values representable by an extension layer-ID, the base layer ID and extension layer ID indexing the layers the portions of the multi-layer data stream are associated with. In accordance with this concept, emphasis may be shifted between increased diversity of the signalizable inter-layer dependencies on the one hand and reduced side-information overhead for signaling the inter-layer dependencies on the other hand.
1. A device configured to process a data stream representing a multi-layered signal into which a video is coded using inter-layer prediction as different layers, which correspond to different levels of information amount, the data stream comprising a plurality of packets, each of which is associated with one of the different layers, each layer being indexed by a base layer-ID or a base layer-ID and an extension layer-ID, the device configured to:
read, from the data stream,
a first inter-dependency syntax structure indicating inter-dependencies between pairs of values representable by the base layer-ID, and
a second inter-dependency syntax structure indicating inter-dependencies between pairs of values representable by the extension layer-ID, wherein an instantiation of the second inter-dependency syntax structure indicates (a) a first layer dependent on, in terms of inter-layer prediction, a second layer, wherein the first and second layers are associated with a same base layer-ID, or (b) a first layer dependent on, in terms of inter-layer prediction, a second layer, wherein the first and second layers are associated with different base layer-IDs and different extension layer-IDs; and
construct, based on the first and second inter-dependency syntax structures, an inter-layer dependency description revealing possible inter-layer prediction dependencies between the different layers.
2. The device according to claim 1 , wherein the device is configured to:
read a plurality of instantiations of the second inter-dependency structure from the data stream, namely for different pairs (j,k) of values 0<k≤j≤n<N representable by the base layer-ID, with N being the number of values representable by the base layer-ID, by
reading, or suppressing reading, the second inter-dependency structure from the data stream for pair (j,k) conditionally depending on the first inter-dependency syntax structure indicating a presence or non-presence of interdependency between pair (j,k),
wherein one of the plurality of instantiations of the second inter-dependency structure read for a pair (j,k) with j=k, indicates which layers among those comprising base layer-ID j are inter-layer prediction dependent on other layers among those comprising base layer-ID j, and another of the plurality of instantiations of the second inter-dependency structure read for pair (j,k), with k<j indicates which layers among those comprising base layer-ID j are inter-layer prediction dependent on layers among those comprising base layer-ID k.
3. The device according to claim 1 , wherein the device is configured to:
read instantiations of the second inter-dependency structure from the data stream, along with, for each instantiation of the second inter-dependency structure, an association syntax structure which associates the respective instantiation of the second inter-dependency structure with a respective subset of pairs (j,k) of values 0<k≤j≤n<N representable by the base layer-ID, with N being the number of values representable by the base layer-ID.
4. The device according to claim 3 , wherein the association syntax structure comprises a flag for each pair (j,k) and the device is configured to:
parse the association syntax structure with skipping pairs (j,k) of values 0<k≤j≤n<N for which the first inter-dependency syntax structure indicates the non-presence of interdependency between pair (j,k).
5. The device according to claim 1 , wherein the number of values representable by the extension layer-ID is M and the second inter-dependency syntax structure comprises
one binary value for each pair (p,q) of extension values 0<q<p≤m<M, in order to indicate which layers among those comprising base layer-ID j are inter-layer prediction dependent on other layers among those comprising base layer-ID j, and
one binary value for each pair (p,q) of extension values 0<p,q≤m<M, in order to indicate which layer with extension value p and comprising base layer-ID j is inter-layer prediction dependent on layer with extension value q and comprising base layer-ID k<j.
6. The device according to claim 1 , wherein the device is configured to derive n from an explicitly signaling of a maximally assumed value among the N values representable by the base layer-ID, in the data stream, and/or derive m from an explicitly signaling of a maximally assumed value among the M values representable by the extension layer-ID, in the data stream.
7. The device according to claim 1 , where the first inter-dependency syntax structure defines interdependencies between different sets of layers, each of the different sets being associated with a unique base layer-ID.
8. The device according to claim 1 , wherein the device is configured to discard packets in units of sets of layers each comprising a common base layer-ID based on the first inter-dependency syntax structure and the packets' base layer-ID.
9. The device according to claim 8 , wherein the device is configured to discard further individual layers based on the first and second inter-dependency syntax structure and the packets' base layer-ID and the packets' extension layer-ID.
10. The device according to claim 1 , wherein the device is
a video decoder configured to decode the multi-layered video signal using the inter-layer prediction and the inter-layer dependency matrix.
11. The device according to claim 1 , wherein the multi-layered video signal has the video coded thereinto at the different layers using inter-layer prediction such that any layer inter-layer predicted from another layer adds one of:
further views;
depth information;
alpha blending information;
color component information;
spatial resolution refinement; and
SNR resolution refinement.
12. A method for processing a data stream representing a multi-layered signal into which a video is coded using inter-layer prediction as different layers, which correspond to different levels of information amount, data stream comprising a plurality of packets, each of which is associated with one of the different layers, each layer being indexed by a base layer-ID or a base layer-ID and an extension layer-ID, the method comprising:
reading, from the data stream,
a first inter-dependency syntax structure indicating interdependencies between pairs of values representable by the base layer-ID, and
a second inter-dependency syntax structure indicating inter-dependencies between pairs of values representable by the extension layer-ID, wherein an instantiation of the second inter-dependency syntax structure indicates (a) a first layer dependent on, in terms of inter-layer prediction, a second layer, wherein the first and second layers are associated with a same base layer-ID, or (b) a first layer dependent on, in terms of inter-layer prediction, a second layer, wherein the first and second layers are associated with different base layer-IDs and different extension layer-IDs; and
constructing, based on the first and second inter-dependency syntax structures, an inter-layer dependency description revealing possible inter-layer prediction dependencies between the different layers.
13. The method according to claim 12 , further comprising:
reading a plurality of instantiations of the second inter-dependency structure from the data stream, namely for different pairs (j,k) of values 0<k≤j≤n<N representable by the base layer-ID, with N being the number of values representable by the base layer-ID, by
reading, or suppressing reading, the second inter-dependency structure from the data stream for pair (j,k) conditionally depending on the first inter-dependency syntax structure indicating a presence or non-presence of interdependency between pair (j,k),
wherein one of the plurality of instantiations of the second inter-dependency structure read for a pair (j,k) with j=k, indicates which layers among those comprising base layer-ID j are inter-layer prediction dependent on other layers among those comprising base layer-ID j, and another of the plurality of instantiations of the second inter-dependency structure read for pair (j,k), with k<j indicates which layers among those comprising base layer-ID j are inter-layer prediction dependent on layers among those comprising base layer-ID k.
14. The method according to claim 12 , further comprising:
reading instantiations of the second inter-dependency structure from the data stream, along with, for each instantiation of the second inter-dependency structure, an association syntax structure which associates the respective instantiation of the second inter-dependency structure with a respective subset of pairs (j,k) of values 0<k≤j≤n<N representable by the base layer-ID, with N being the number of values representable by the base layer-ID.
15. The method according to claim 14 , wherein the association syntax structure comprises a flag for each pair (j,k) and the method further comprises:
parsing the association syntax structure with skipping pairs (j,k) of values 0<k≤j≤n<N for which the first inter-dependency syntax structure indicates the non-presence of interdependency between pair (j,k).
16. The method according to claim 12 , wherein the number of values representable by the extension layer-ID is M and the second inter-dependency syntax structure comprises
one binary value for each pair (p,q) of extension values 0<q<p≤m<M, in order to indicate which layers among those comprising base layer-ID j are inter-layer prediction dependent on other layers among those comprising base layer-ID j, and
one binary value for each pair (p,q) of extension values 0<p,q≤m<M, in order to indicate which layer with extension value p and comprising base layer-ID j is inter-layer prediction dependent on layer with extension value q and comprising base layer-ID k<j.
17. The method according to claim 12 , further comprising discarding packets in units of sets of layers each comprising a common base layer-ID based on the first inter-dependency syntax structure and the packets' base layer-ID.
18. The method according to claim 17 , further comprising discarding further individual layers based on the first and second inter-dependency syntax structure and the packets' base layer-ID and the packets' extension layer-ID.
19. An encoder configured to encode into a data stream representing a multi-layered signal into which a video using inter-layer prediction as different layers, which correspond to different levels of information amount, the data stream comprising a plurality of packets, each of which is associated with one of the different layers, each layer being indexed by a base layer-ID or a base layer-ID and an extension layer-ID, the encoder configured to:
insert, into the data stream,
a first inter-dependency syntax structure indicating interdependencies between pairs of values representable by the base layer-ID, and
a second inter-dependency syntax structure indicating inter-dependencies between pairs of values representable by the extension layer-ID, wherein an instantiation of the second inter-dependency syntax structure indicates (a) a first layer dependent on, in terms of inter-layer prediction, a second layer, wherein the first and second layers are associated with a same base layer-ID, or (b) a first layer dependent on, in terms of inter-layer prediction, a second layer, wherein the first and second layers are associated with different base layer-IDs and different extension layer-IDs,
with setting the first and second inter-dependency syntax structures such that, based on the first and second inter-dependency syntax structures, an inter-layer dependency description revealing possible inter-layer prediction dependencies between the different layers is constructible.
20. The encoder according to claim 19 , wherein the multi-layered video signal has the video coded thereinto at the different layers using inter-layer prediction such that any layer inter-layer predicted from another layer adds one of:
further views;
depth information;
alpha blending information;
color component information;
spatial resolution refinement; and
SNR resolution refinement.