Multi-view coding with efficient residual handling
A gain in multi-view coding is achieved as follows: the residual signal involved with coding a dependent view of the multi-view signal is predicted from a reference residual signal of the current picture of the reference view using block-granular disparity-compensated prediction, i.e. using disparity compensated prediction with a disparity defined at, and varying with, block granularity so that each block of the current picture of the dependent view has its own disparity displacement such as its own disparity vector, associated therewith. In other words, a remaining similarity between the residual signal involved with predictively coding the reference view is used in order to predict the residual signal involved with predictively coding the dependent view.
1. A method for reconstructing a multi-view signal coded in a data stream, comprising:
identifying a reference-view coding block in a picture of a reference view of a multi-view signal based on a disparity vector, wherein the disparity vector represents a disparity between the reference-view coding block and a dependent coding block in a picture of a dependent view of the multi-view signal;
obtaining a reference-view residual signal associated with the reference-view coding block, wherein the reference-view residual signal represents a difference between the reference-view coding block and a prediction of the reference-view coding block;
estimating a dependent-view residual signal for the dependent coding block in the picture of the dependent view based on the reference-view residual signal using the disparity vector, wherein the dependent-view residual signal represents a difference between the dependent coding block and a prediction of the dependent coding block;
adding a remaining signal corresponding to the dependent-view residual signal to the estimated dependent-view residual signal to obtain a reconstructed dependent-view residual signal; and
reconstructing the dependent coding block based on the reconstructed dependent-view residual signal and a prediction of the dependent coding block.
2. The method of claim 1 , wherein the obtaining the reference-view residual signal includes extracting the reference-view residual signal from the data stream.
3. The method of claim 1 , further comprising:
extracting, from the data stream, a second order prediction residual associated with the dependent coding block;
refining the estimated dependent-view residual signal for the dependent coding block based on the second order prediction residual to generate a refined estimated dependent-view residual signal; and
reconstructing the dependent coding block based on the refined estimated dependent-view residual signal.
4. The method of claim 1 , further comprising: extracting, from the data stream, disparity vector residual associated with the dependent coding block; refining the disparity vector for the dependent coding block based on the disparity vector residual to generate a refined disparity vector; and identifying the reference-view coding block based on the refined disparity vector.
5. The method of claim 1 , wherein the reference-view residual signal is represented in a transform domain, and the estimated dependent-view residual signal is determined in the transform domain.
6. The method of claim 1 , further comprising obtaining the disparity vector including estimating the disparity vector based on temporal prediction using information related to a reference block in a previously reconstructed picture of the dependent view.
7. An apparatus for reconstructing a multi-view signal coded in a data stream, comprising:
a dependent-view reconstructor configured for, using a processor: identifying a reference-view coding block in a picture of a reference view of a multi-view signal based on a disparity vector, wherein the disparity vector represents a disparity between the reference-view coding block and a dependent coding block in a picture of a dependent view of the multi-view signal;
obtaining a reference-view residual signal associated with the reference-view coding block, wherein the reference-view residual signal represents a difference between the reference-view coding block and a prediction of the reference-view coding block;
estimating a dependent-view residual signal for the dependent coding block in the picture of the dependent view based on the reference-view residual signal using the disparity vector, wherein the dependent-view residual signal represents a difference between the dependent coding block and a prediction of the dependent coding block;
adding a remaining signal corresponding to the dependent-view residual signal to the estimated dependent-view residual signal to obtain a reconstructed dependent-view residual signal; and
reconstructing the dependent coding block based on the reconstructed dependent-view residual signal and a prediction of the dependent coding block.
8. The apparatus of claim 7 , further comprising a reference-view reconstructor configured for extracting the reference-view residual signal from the data stream.
9. The apparatus of claim 7 , wherein the dependent-view reconstructor is further configured for:
extracting, from the data stream, a second order prediction residual associated with the dependent coding block;
refining the estimated dependent-view residual signal for the dependent coding block based on the second order prediction residual to generate a refined estimated dependent-view residual signal; and
reconstructing the dependent coding block based on the refined estimated dependent-view residual signal.
10. The apparatus of claim 7 , wherein the dependent-view reconstructor is further configured for:
extracting, from the data stream, disparity vector residual associated with the dependent coding block;
refining the disparity vector for the dependent coding block based on the disparity vector residual to generate a refined disparity vector; and
identifying the reference-view coding block based on the refined disparity vector.
11. The apparatus of claim 7 , wherein the reference-view residual signal is represented in a transform domain, and the estimated dependent-view residual signal is determined in the transform domain.
12. The apparatus of claim 7 , further comprising a disparity estimator configured for obtaining the disparity vector with respect to the dependent coding block by estimating the disparity vector based on temporal prediction using information related to a reference block in a previously reconstructed picture of the dependent view.
13. A method for encoding a multi-view signal into a data stream, comprising:
determining a reference-view residual signal associated with a reference-view coding block in a picture of a reference view of the multi-view signal, wherein the reference-view residual signal is to be used to obtain an estimate of a dependent-view residual signal for a dependent coding block in a picture of a dependent view of the multi-view signal via:
identifying the reference-view coding block based on a disparity vector, wherein the disparity vector represents a disparity between the reference-view coding block and the dependent coding block, obtaining the reference-view residual signal associated with the reference-view coding block, wherein the reference-view residual signal represents a difference between a reference-view coding block and a prediction of the reference-view coding block, estimating the dependent-view residual signal for the dependent coding block in the picture of the dependent view based on the reference-view residual signal using the disparity vector, wherein the dependent-view residual signal represents a difference between the dependent coding block and a prediction of the dependent coding block, and determining a remaining signal that represents a difference between the dependent-view residual signal and the estimated dependent-view residual signal; and
inserting, into the data stream, the reference-view residual signal and the remaining signal.
14. The method of claim 13 , wherein the obtaining the reference-view residual signal includes extracting the reference-view residual signal from the data stream.
15. The method of claim 13 , the estimating the dependent-view residual signal includes:
extracting, from the data stream, a second order prediction residual associated with the dependent coding block;
refining the estimated dependent-view residual signal for the dependent coding block based on the second order prediction residual to generate a refined estimated dependent-view residual signal; and
reconstructing the dependent coding block based on the refined estimated dependent-view residual signal.
16. The method of claim 13 , the estimating the dependent-view residual signal includes:
extracting, from the data stream, disparity vector residual associated with the dependent coding block;
refining the disparity vector for the dependent coding block based on the disparity vector residual to generate a refined disparity vector; and
identifying the reference-view coding block based on the refined disparity vector.
17. The method of claim 13 , wherein the reference-view residual signal is represented in a transform domain; and the estimated dependent-view residual signal is determined in the transform domain.
18. The method of claim 13 , further comprising obtaining the disparity vector including estimating the disparity vector based on temporal prediction using information related to a reference block in a previously reconstructed picture of the dependent view.
19. A machine readable non-transitory medium for storing data associated with a multi-view signal, comprising:
a data stream stored in the non-transitory machine readable medium, the data stream comprising a remaining signal and a reference-view residual signal associated with a picture of a reference view of the multi-view signal, wherein the reference-view residual signal is to be used to obtain an estimate of a dependent-view residual signal for a dependent coding block in a picture of a dependent view of the multi-view signal via: identifying a reference-view coding block in a picture of a reference view based on a disparity vector, wherein the disparity vector represents a disparity between the reference-view coding block and the dependent coding block, obtaining the reference-view residual signal associated with the reference-view coding block, wherein the reference-view residual signal represents a difference between a reference-view coding block and a prediction of the reference-view coding block, estimating the dependent-view residual signal for the dependent coding block in the picture of the dependent view based on the reference-view residual signal using the disparity vector, wherein the dependent-view residual signal represents a difference between the dependent coding block and a prediction of the dependent coding block, and determining a remaining signal that represents a difference between the dependent-view residual signal and the estimated dependent-view residual signal; and
reconstructing the dependent coding block based on the reconstructed dependent-view residual signal and a prediction of the dependent coding block.
20. The machine readable non-transitory medium of claim 19 , wherein the disparity vector is obtained by estimating the disparity vector based on temporal prediction using information related to a reference block in a previously reconstructed picture of the dependent view.