Hybrid video coding supporting intermediate view synthesis
Hybrid video decoder supporting intermediate view synthesis of an intermediate view video from a first- and a second-view video which are predictively coded into a multi-view data signal with frames of the second-view video being spatially subdivided into sub-regions and the multi-view data signal having a prediction mode is provided, having: an extractor configured to respectively extract, from the multi-view data signal, for sub-regions of the frames of the second-view video, a disparity vector and a prediction residual; a predictive reconstructor configured to reconstruct the sub-regions of the frames of the second-view video, by generating a prediction from a reconstructed version of a portion of frames of the first-view video using the disparity vectors and a prediction residual for the respective sub-regions; and an intermediate view synthesizer configured to reconstruct first portions of the intermediate view video.
1. A non-transitory computer-readable medium for storing video data, comprising:
a data stream stored in the non-transitory computer-readable medium and comprising multi-view data including at least a first-view video and a second-view video, wherein frames of the second-view video are spatially subdivided into sub-regions, each of which is associated with a prediction mode selected from at least an inter-view prediction mode and an intra-view prediction mode,
for each of the sub-regions, the multi-view data include an associated inter-view prediction mode, a disparity vector, a prediction residual, and reliability data associated with the disparity vector, where
the disparity vector is selected from a plurality of disparity vectors determined within a predetermined search area,
the prediction residual is determined based on the selected disparity vector, and
data associated with a portion of a frame of the first-view video and the disparity vector are used to synthesize a first portion of a frame of a third-view video.
2. The non-transitory computer-readable medium of claim 1 , wherein the first portion of the frame of the third-view video is used to extrapolate or interpolate a second portion of the frame of the third-view video.
3. The non-transitory computer-readable medium of claim 1 , wherein, based on the disparity vector, sample positions of one of the sub-region of the frame of the second-view video are linearly mapped into the first-view video, and the portion of the frame of the first-view video is sampled at the sample positions to determine the prediction for the one of the sub-regions.
4. The non-transitory computer-readable medium of claim 1 , wherein, based on the disparity vector, sample positions of one of the sub-region of the frame of the second-view video are linearly mapped into a direction opposite to the disparity vector, and
the one of the sub-regions is sampled at the sample positions, with a reduction of an amount of the linear mapping based on a spatial location of a third view corresponding to the third-view video, relative to a first view corresponding to the first-view video and a second view corresponding to the second-view video.
5. The non-transitory computer-readable medium of claim 1 , wherein the disparity vector is excluded in synthesizing the first portion of the third-view video if the reliability data fails to satisfy a predetermined requirement.
6. An apparatus comprising a non-transitory computer-readable medium for storing data provided by an encoder, the computer-readable medium comprising:
a data stream stored in the computer-readable medium, and comprising multi-view data including at least a first-view video and a second-view video, wherein frames of the second-view video are spatially subdivided into sub-regions,
the encoder associates each sub-region with a prediction mode selected from at least an inter-view prediction mode and an intra-view prediction mode, and
the encoder, for each of the sub-regions associated with the inter-view prediction mode,
selects a disparity vector from a plurality of disparity vectors determined within a predetermined search area,
determines a prediction residual based on the selected disparity vector, and
inserts, into the multi-view, data the disparity vector, the prediction residual, and reliability data associated with the disparity vector,
wherein data associated with a portion of a frame of the first-view video and the disparity vector are used to synthesize a portion of a frame of a third-view video.