Efficient multi-view coding using depth-map estimate for a dependent view
The usual coding order according to which the reference view is coded prior to the dependent view, and within each view, a depth map is coded subsequent to the respective picture, may be maintained and does lead to a sacrifice of efficiency in performing inter-view redundancy removal by, for example, predicting motion data of the current picture of the dependent view from motion data of the current picture of the reference view. Rather, a depth map estimate of the current picture of the dependent view is obtained by warping the depth map of the current picture of the reference view into the dependent view, thereby enabling various methods of inter-view redundancy reduction more efficiently by bridging the gap between the views. According to another aspect, the following discovery is exploited: the overhead associated with an enlarged list of motion predictor candidates for a block of a picture of a dependent view is comparatively low compared to a gain in motion vector prediction quality resulting from an adding of a motion vector candidate which is determined from an, in disparity-compensated sense, co-located block of a reference view.
1. An apparatus for reconstructing a multi-view signal coded into a data stream, comprising:
a depth estimator configured to obtain, using a processor, a depth map estimate of a current picture in a dependent view of the multi-view signal based on information from the data stream; and
a dependent-view reconstructor configured to, using the processor:
determine a disparity vector for at least one block of blocks of the current picture in the dependent view of the multi-view signal based on the depth map estimate, the disparity vector representing a disparity between the current picture of the dependent view and a current picture of a reference view of the multi-view signal at the at least one block of the current picture of the dependent view;
derive, for the at least one block of the current picture of the dependent view, a list of motion vector predictor candidates by:
determining a block within the current picture of the reference view using the disparity vector, and
adding a motion vector to the list of motion vector predictor candidates based on information associated with the determined block of the current picture of the reference view; and
identify, for the at least one block of the current picture of the dependent view, a motion vector predictor candidate from the list of motion vector predictor candidates; and
reconstruct the at least one block of the current picture of the dependent view by performing a motion-compensated prediction of the at least one block of the current picture of the dependent view using a motion vector which depends on the identified motion vector predictor candidate.
2. The apparatus according to claim 1 , wherein the dependent-view reconstructor is configured to extract, for the at least one block of the current picture of the dependent view, from the data stream index information specifying the motion vector predictor candidate from the list of motion vector predictor candidates.
3. The apparatus according to claim 1 , further comprising:
a reference-view reconstructor configured to reconstruct a depth map of the current picture of the reference view of the multi-view signal from a reference view depth-map portion of the multi-view data stream, wherein:
the depth estimator is configured to estimate the depth map of the current picture of the dependent view by warping the depth map of the current picture of the reference view into the dependent view, and
the dependent-view reconstructor is configured to, in determining the disparity vector for the at least one block, subject the depth data estimate at the at least one block to depth-to-disparity conversion to acquire the disparity vector.
4. The apparatus according to claim 1 , wherein the dependent-view reconstructor is configured to extract, for the at least one block of the current picture of the dependent view, a motion vector difference in relation to the identified motion vector predictor candidate and to perform the reconstruction of the at least one block of the current picture such that the used motion vector further depends on a sum of the motion vector difference and the identified motion vector predictor candidate.
5. The apparatus according to claim 1 , wherein the dependent-view reconstructor is configured to extract for the at least one block of the current picture of the dependent view, a reference picture index specifying a reference picture of a list of reference pictures comprising the current picture of the reference view and already decoded pictures of the dependent view,
wherein the dependent-view reconstructor is configured to, if the reference picture is one of the already decoded pictures of the dependent view, perform the motion-compensated prediction using the one already decoded picture of the dependent view as a reference, and if the reference picture is the current picture of the reference view, add the disparity vector or a modified disparity vector derived from the disparity vector to a list of disparity vector prediction candidates, extract index information specifying one disparity vector predictor candidate of the list of disparity vector predictor candidates from the multi-view data stream and reconstruct the at least one block of the current picture of the dependent view by performing a disparity-compensated prediction of the at least one block of the current picture of the dependent view using a disparity vector which depends on the specified disparity vector candidate using the current picture of the reference view as a reference.
6. The apparatus according to claim 1 , wherein the dependent-view reconstructor is further configured to, in deriving the list of motion vector predictor candidates, spatially and/or temporarily predict a further motion vector from spatially and/or temporarily neighboring blocks of the dependent view and add the further motion vector or a version derived from the further motion vector to the list of motion vector predictor candidates.
7. The apparatus according to claim 1 , wherein the dependent-view reconstructor is configured to perform the derivation of the list of motion vector predictor candidates via a list of motion/disparity vector predictor candidates being a list of motion/disparity parameter candidates each comprising a number of hypotheses, and, per hypothesis, a motion/disparity motion vector and a reference index specifying a reference picture out of a list of reference pictures comprising the current picture of a reference view and previously decoded pictures of the dependent view, wherein the dependent-view reconstructor is configured to add motion/disparity parameters to the list of motion/disparity parameter candidates which depend on motion/disparity parameters associated with the determined block of the current picture of the reference view, and to reconstruct the at least one block of the current picture of the dependent view by performing motion/disparity-compensated prediction on the at least one block of the current picture of the dependent view using motion/disparity parameters which depend on a motion/disparity parameter candidate specified by the index information.
8. An apparatus for encoding a multi-view signal into a data stream, comprising:
a depth estimator configured to obtain, using a processor, a depth map estimate of a current picture in a dependent view of the multi-view signal; and
a dependent-view encoder configured to, using the processor:
determine a disparity vector for at least one block of blocks of the current picture in the dependent view of the multi-view signal based on the depth map estimate, the disparity vector representing a disparity between the current picture of the dependent view and a current picture of a reference view of the multi-view signal at the at least one block of the current picture of the dependent view;
derive, for the at least one block of the current picture of the dependent view, a list of motion vector predictor candidates by:
determining a block within the current picture of the reference view using the disparity vector, and
adding a motion vector to the list of motion vector predictor candidates which depends on information associated with the determined block of the picture of the reference view; and
encode the at least one block of the current picture of the dependent view by performing a motion-compensated prediction of the at least one block of the current picture of the dependent view using a motion vector which depends on a certain motion vector predictor candidate from the list of motion vector predictor candidates.
9. The apparatus according to claim 8 , wherein the dependent-view encoder is configured to insert, into the data stream, index information specifying the certain motion vector predictor candidate from the list of motion vector predictor candidates for the at least one block of the current picture of the dependent view.
10. The apparatus according to claim 8 , further comprising:
a reference-view encoder configured to encode a depth map of the current picture of the reference view of the multi-view signal into a reference view depth-map portion of the multi-view data stream, wherein:
the depth estimator is configured to estimate the depth map of the current picture of the dependent view by warping the depth map of the current picture of the reference view into the dependent view, and
the dependent-view encoder is configured to, in determining the disparity vector for the at least one block, subject the depth data estimate at the at least one block to depth-to-disparity conversion to acquire the disparity vector.
11. A method for encoding a multi-view signal into a data stream, comprising:
obtaining a depth map estimate of a current picture in a dependent view of the multi-view signal;
determining a disparity vector for at least one block of blocks of the current picture in the dependent view of the multi-view signal based on the depth map estimate, the disparity vector representing a disparity between the current picture of the dependent view and a current picture of a reference view of the multi-view signal at the at least one block of the current picture of the dependent view;
deriving, for the at least one block of the current picture of the dependent view, a list of motion vector predictor candidates by:
determining a block within the current picture of the reference view using the disparity vector, and
adding a motion vector to the list of motion vector predictor candidates which depends on information associated with the determined block of the picture of the reference view; and
encoding the at least one block of the current picture of the dependent view by performing a motion-compensated prediction of the at least one block of the current picture of the dependent view using a motion vector which depends on a certain motion vector predictor candidate from the list of motion vector predictor candidates.
12. The method according to claim 11 , further comprising:
inserting, into the data stream, index information specifying the certain motion vector predictor candidate from the list of motion vector predictor candidates for the at least one block of the current picture of the dependent view.
13. The method according to claim 11 , further comprising:
encoding a depth map of the current picture of the reference view of the multi-view signal into a reference view depth-map portion of the multi-view data stream, wherein the depth map of the current picture of the dependent view is estimated by warping the depth map of the current picture of the reference view into the dependent view; and
in determining the disparity vector for the at least one block, subjecting the depth data estimate at the at least one block to depth-to-disparity conversion to acquire the disparity vector.
14. A method for storing a data stream, comprising storing on a digital storage medium, a data stream into which a multi-view signal is encoded by the method according to claim 11 .