Hybrid video coding supporting intermediate view synthesis

1. A hybrid video decoder comprising:

an extractor configured to:

receive a multi-view data comprising information on a first view video and a second view video;

determine, from the received multi-view data, a disparity vector and a prediction residual associated with a sub-region of a frame of the second view video, wherein the sub-region of the second view video is associated with an inter-view prediction mode; and

generate, based on the multi-view data, a reconstructed version of a portion of a frame of the first view video based on an inverse transformation;

a predictive reconstructor configured to:

receive, from the extractor, the reconstructed version of the portion of the frame of the first view video; and

reconstruct the sub-region of the frame of the second view video from the reconstructed version of the portion of the frame of the first view video using the disparity vector and the prediction residual extracted from the multi-view data; and

a view synthesizer configured to reconstruct a first portion of a frame of a third view video using the reconstructed version of the portion of the frame of the first view video and the disparity vector.

2. The hybrid video decoder according to claim 1 , wherein the view synthesizer is configured to use the reconstruction of the first portion of the frame of the third view video as supporting points in extra- and/or interpolating a second portion of the third view video.

3. The hybrid video decoder according to claim 1 , wherein the view synthesizer is configured to finish a reconstruction of the third view video group-of-picture wise.

4. The hybrid video decoder according to claim 1 , wherein the predictive reconstructor and the view synthesizer are configured to:

linearly map, based on the disparity vector, sample positions of the sub-region of the frame of the second view video into the first view video; and

sample the reconstructed version of the portion of the frame of the first view video at the sample positions to acquire the prediction for the sub-region of the frame of the second view video and the reconstruction of the first portion of the third view video.

5. The hybrid video decoder according to claim 1 ,

wherein the predictive reconstructor is configured to:

linearly map, based on the disparity vector, sample positions of the sub-region of the frame of the second view video into the first view video; and

sample the reconstructed version of the portion of the frame of the first view video at the sample positions to acquire the prediction for the sub-region of the frame of the second view video; and

wherein the view synthesizer is configured to:

linearly map, based on the disparity vector, sample positions of the sub-region of the frame of the second view video into a direction opposite to the disparity vector; and

sample the reconstruction of the sub-region of the frame of the second view video at the sample positions thus acquired, with a reduction of an amount of the linear mapping of the view synthesizer relative to the predictive reconstructor, which depends on a spatial location of a third view corresponding to the third view video, relative to a first view corresponding to the first view video, and a second view corresponding to the second view video, respectively.

6. The hybrid video decoder according to claim 1 , wherein the extractor is further configured to extract, from the multi-view data, reliability data associated with the disparity vector, wherein the view synthesizer is configured to exclude the disparity vector in reconstructing the first portion of the third view video when the reliability data fails to satisfy a predetermined requirement.

7. A hybrid video decoding method comprising:

receiving, by an extractor of a hybrid video decoder, a multi-view data comprising information on a first view video and a second view video;

extracting, by the extractor, from the multi-view data, a disparity vector and a prediction residual associated with a sub-region of a frame of the second view video, wherein the sub-region of the second view video is associated with an inter-view prediction mode;

generating, by the extractor, based on the multi-view data, a reconstructed version of a portion of a frame of the first view video using an inverse transformation;

reconstructing, by a predictive reconstructor of the hybrid video decoder, the sub-region of the frame of the second view video from the reconstructed version of the portion of the frame of the first view video using the disparity vector and the prediction residual extracted from the multi-view data; and

reconstructing, by a view synthesizer of the hybrid video decoder, a first portion of a frame of a third view video using the reconstructed version of the portion of the frame of the first view video and the disparity vector.

8. A hybrid video encoder configured to encode a first- and a second-view video into a multi-view data, wherein frames of the second view video are spatially subdivided into sub-regions, the hybrid video encoder comprising:

a predictive reconstructor configured to:

select a prediction mode from a plurality of prediction modes for each of the sub-regions of the frames of the second view video, wherein the plurality of prediction modes comprises at least an inter-view prediction mode and an intra-view prediction mode;

respectively determine, for the sub-regions of the frames of the second view video associated with the inter-view prediction mode, (1) a disparity vector selected from a plurality of disparity vectors lying within a predetermined search area, (2) the prediction residual for the respective sub-regions, resulting from applying the determined disparity vector, and (3) a reliability data associated with the disparity vector; and

a data signal generator configured to respectively insert, for the sub-regions of the frames of the second view video associated with the inter-view prediction mode, the determined disparity vector, the determined prediction residual, and the determined reliability data into the multi-view data.

9. A video encoding method for encoding a first- and a second-view video into a multi-view data, wherein frames of the second view video are spatially subdivided into sub-regions, the method comprising:

selecting, by a predictive reconstructor of a video encoder, a prediction mode from a plurality of prediction modes for each of the sub-regions of the frames of the second view video, wherein the plurality of prediction modes comprises at least an inter-view prediction mode and an intra-view prediction mode;

respectively determining, by the predictive reconstructor, for the sub-regions of the frames of the second view video associated with the inter-view prediction mode, (1) a disparity vector selected from a plurality of disparity vectors lying within a predetermined search area, (2) the prediction residual for the respective sub-regions, resulting from applying the disparity vector determined, and (3) a reliability data associated with the disparity vector; and

respectively inserting, by a data signal generator of the video encoder, for the sub-regions of the frames of the second view video associated with the inter-view prediction mode, the determined disparity vector, the determined prediction residual, and the determined reliability data into the multi-view data.

10. A non-transitory computer readable medium including a computer program comprising a program code, wherein the program code is operable to cause a computer to perform a hybrid video decoding method, the method comprising:

receiving, by an extractor of a hybrid video decoder, a multi-view data comprising information on a first view video and a second view video;

extracting, by the extractor, from the multi-view data, a disparity vector and a prediction residual associated with a sub-region of a frame of the second view video, wherein the sub-region of the second view video is associated with an inter-view prediction mode;

generating, by the extractor, based on the multi-view data, a reconstructed version of a portion of a frame of the first view video using an inverse transformation;

reconstructing, by a predictive reconstructor of the hybrid video decoder, the sub-region of the frame of the second view video from the reconstructed version of the portion of the frame of the first view video using the disparity vector and the prediction residual extracted from the multi-view data; and

reconstructing, by a view synthesizer of the hybrid video decoder, a first portion of a frame of a third view video using the reconstructed version of the portion of the frame of the first view video and the disparity vector.

11. A non-transitory computer readable medium including a computer program comprising a program code, wherein the program code is operable to cause a computer to perform a method for encoding a first- and a second-view video into a multi-view data, wherein frames of the second view video are spatially subdivided into sub-regions, the method comprising:

selecting, by a predictive reconstructor, a prediction mode from a plurality of prediction modes for each of the sub-regions of the frames of the second view video, wherein the plurality of prediction modes comprises at least an inter-view prediction mode and an intra-view prediction mode,

respectively determining, by the predictive reconstructor, for the sub-regions of the frames of the second view video associated with the inter-view prediction mode, (1) a disparity vector selected from a plurality of disparity vectors lying within a predetermined search area, (2) the prediction residual for the respective sub-regions, resulting from applying the determined disparity vector, and (3) a reliability data associated with the disparity vector; and

respectively inserting, by a data signal generator, for the sub-regions of the frames of the second view video associated with the inter-view prediction mode, the determined disparity vector, the determined prediction residual, and the determined reliability data into the multi-view data.