Efficient multi-view coding using depth-map estimate and update
This disclosure is directed to coding a multi-view signal, which includes processing a list of plurality of motion vector candidates associated with a coding block of a current picture in a dependent view of the multi-view signal. Such processing includes estimating a first motion vector based on a second motion vector associated with a reference block in a current picture of a reference view of the multi-view signal, the reference block corresponding to the coding block of the current picture in the dependent view. The first motion vector is added into the list, and an index is used that specifies at least one candidate from the list to be used for motion-compensated prediction. The coding block in the current picture is coded by performing the motion-compensated prediction based on the at least one candidate indicated by the index.
1. An apparatus for reconstructing a multi-view signal coded in a data stream, comprising:
a dependent view reconstructor configured for:
processing a list of plurality of motion vector candidates associated with a coding block of a current picture in a dependent view of the multi-view signal, the processing including:
estimating a first motion vector based on a second motion vector associated with a reference block in a current picture of a reference view of the multi-view signal, the reference block corresponding to the coding block of the current picture in the dependent view,
adding the first motion vector into the list, and
extracting, from the data stream, index information that specifies at least one candidate from the list to be used for motion-compensated prediction; and
reconstructing the coding block in the current picture of the dependent view by performing the motion-compensated prediction based on the at least one candidate.
2. The apparatus of claim 1 , wherein the dependent view reconstructor is configured for obtaining a disparity vector with respect to the coding block of the current picture in the dependent view, wherein the disparity vector represents a disparity between the current picture of the dependent view and the current picture of the reference view, and identifying the reference block in the current picture of the reference view based on the disparity vector.
3. The apparatus of claim 2 , further comprising a depth estimator configured for:
obtaining motion data associated with the current picture of the reference view;
applying the motion data associated with the current picture of the reference view to a depth map estimate of a previous picture of the reference view to generate a first estimated depth map for the current picture of the reference view;
deriving a second estimated depth map for the current picture of the dependent view based on the estimated depth map for the current picture of the reference view; and
determining the disparity vector based on the second estimated depth map for the current picture of the dependent view.
4. The apparatus of claim 3 , wherein the disparity vector for the coding block in the dependent view is determined further based on at least one disparity vector associated with one or more previously decoded coding blocks of the multi-view signal.
5. The apparatus of claim 1 , wherein the dependent-view reconstructor is further configured for:
extracting, from the data stream, a motion vector residual directed to the selected motion vector;
obtaining a refined motion vector based on the selected motion vector and the motion vector residual; and
performing the motion-compensated prediction to predict the coding block based on the refined motion vector.
6. The apparatus of claim 1 , wherein the list includes one or more of:
a first estimated motion vector determined based on motion data associated with a spatial neighbor coding block in the current picture of the dependent view,
a second estimated motion vector determined based on motion data associated with a temporal neighbor coding block in a previous picture of the dependent view,
a first modified estimated motion vector derived by modifying the first estimated motion vector, and
a second modified estimated motion vector derived by modifying the second estimated motion vector.
7. A non-transitory machine-readable medium having information stored thereon for reconstructing a multi-view signal coded in a data stream, wherein the information, when read by the machine, cause the machine to perform a plurality of operations comprising:
processing a list of plurality of motion vector candidates associated with a coding block of a current picture in a dependent view of the multi-view signal, comprising:
estimating a first motion vector based on a second motion vector associated with a reference block in a current picture of a reference view of the multi-view signal, the reference block corresponding to the coding block of the current picture in the dependent view,
adding the first motion vector into the list, and
extracting, from the data stream, index information that specifies at least one candidate from the list to be used for motion-compensated prediction; and
reconstructing the coding block in the current picture of the dependent view by performing the motion-compensated prediction based on the at least one candidate.
8. The non-transitory machine-readable medium of claim 7 , the plurality of operations further comprising:
obtaining a disparity vector with respect to the coding block of the current picture in the dependent view, wherein the disparity vector represents a disparity between the current picture of the dependent view and the current picture of the reference view; and
identifying the reference block in the current picture of the reference view based on the disparity vector.
9. The non-transitory machine-readable medium of claim 8 , wherein the obtaining the disparity vector comprises:
obtaining motion data associated with the current picture of the reference view;
applying the motion data associated with the current picture of the reference view to a depth map estimate of a previous picture of the reference view to generate a first estimated depth map for the current picture of the reference view;
deriving a second estimated depth map for the current picture of the dependent view based on the estimated depth map for the current picture of the reference view; and
determining the disparity vector based on the second estimated depth map for the current picture of the dependent view.
10. The non-transitory machine-readable medium of claim 9 , wherein the disparity vector for the coding block in the dependent view is determined further based on at least one disparity vector associated with one or more previously decoded coding blocks of the multi-view signal.
11. The non-transitory machine-readable medium of claim 7 , the plurality of operations further comprising:
extracting, from the data stream, a motion vector residual directed to the selected motion vector;
obtaining a refined motion vector based on the selected motion vector and the motion vector residual; and
performing the motion-compensated prediction to predict the coding block based on the refined motion vector.
12. The non-transitory machine-readable medium of claim 7 , wherein the list includes one or more of:
a first estimated motion vector determined based on motion data associated with a spatial neighbor coding block in the current picture of the dependent view,
a second estimated motion vector determined based on motion data associated with a temporal neighbor coding block in a previous picture of the dependent view,
a first modified estimated motion vector derived by modifying the first estimated motion vector, and
a second modified estimated motion vector derived by modifying the second estimated motion vector.
13. An apparatus for encoding a multi-view signal coded into a data stream, comprising:
a dependent view encoder configured for
processing a list of plurality of motion vector candidates associated with a coding block of a current picture in a dependent view of the multi-view signal, comprising:
estimating a first motion vector based on a second motion vector associated with a reference block in a current picture of a reference view of the multi-view signal, the reference block corresponding to the coding block of the current picture in the dependent view,
adding the first motion vector into the list,
determining at least one candidate from the list to be used for reconstructing the coding block in the current picture of the dependent view via motion-compensated prediction, and
generating index information specifying the at least one candidate; and
inserting, into the data stream, the index information.
14. The apparatus of claim 13 , wherein the dependent view encoder is configured for obtaining a disparity vector with respect to the coding block of the current picture in the dependent view, wherein the disparity vector represents a disparity between the current picture of the dependent view and the current picture of the reference view, and identifying the reference block in the current picture of the reference view based on the disparity vector.
15. The apparatus of claim 14 , further comprising a depth estimator configured for estimating the disparity vector by:
obtaining motion data associated with the current picture of the reference view;
applying the motion data associated with the current picture of the reference view to a depth map estimate of a previous picture of the reference view to generate a first estimated depth map for the current picture of the reference view;
deriving a second estimated depth map for the current picture of the dependent view based on the estimated depth map for the current picture of the reference view; and
determining the disparity vector based on the second estimated depth map for the current picture of the dependent view.
16. The apparatus of claim 15 , wherein the disparity vector for the coding block in the dependent view is determined further based on at least one disparity vector associated with one or more previously decoded coding blocks of the multi-view signal.
17. The apparatus of claim 13 , wherein the dependent-view encoder is further configured for:
extracting, from the data stream, a motion vector residual directed to the selected motion vector;
obtaining a refined motion vector based on the selected motion vector and the motion vector residual; and
performing the motion-compensated prediction to predict the coding block based on the refined motion vector.
18. The apparatus of claim 13 , wherein the list includes one or more of:
a first estimated motion vector determined based on motion data associated with a spatial neighbor coding block in the current picture of the dependent view,
a second estimated motion vector determined based on motion data associated with a temporal neighbor coding block in a previous picture of the dependent view,
a first modified estimated motion vector derived by modifying the first estimated motion vector, and
a second modified estimated motion vector derived by modifying the second estimated motion vector.