Sub-prediction unit (PU) based temporal motion vector prediction in HEVC and sub-PU design in 3D-HEVC
Techniques are described for sub-prediction unit (PU) based motion prediction for video coding in HEVC and 3D-HEVC. In one example, the techniques include an advanced temporal motion vector prediction (TMVP) mode to predict sub-PUs of a PU in single layer coding for which motion vector refinement may be allowed. The advanced TMVP mode includes determining motion vectors for the PU in at least two stages to derive motion information for the PU that includes different motion vectors and reference indices for each of the sub-PUs of the PU. In another example, the techniques include storing separate motion information derived for each sub-PU of a current PU predicted using a sub-PU backward view synthesis prediction (BVSP) mode even after motion compensation is performed. The additional motion information stored for the current PU may be used to predict subsequent PUs for which the current PU is a neighboring block.
1. A method of processing video data comprising:
determining a first stage motion vector for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU as a first stage of an advanced temporal motion vector prediction (TMVP) mode, wherein the first stage motion vector identifies a block of a reference picture corresponding to the PU;
partitioning the PU into two or more sub-PUs;
determining second stage motion information for each of the sub-PUs from the block of the reference picture identified by the first stage motion vector as a second stage of the advanced TMVP mode, wherein the second stage motion information for each of the sub-PUs includes at least one motion vector and an associated reference index; and
performing motion compensation for each of the sub-PUs separately based on the second stage motion information for each of the sub-PUs.
2. The method of claim 1 , wherein determining the first stage motion vector for the PU comprises deriving the first stage motion vector from one or more of spatial neighboring blocks or temporal neighboring blocks of the PU.
3. The method of claim 1 , wherein determining the first stage motion vector for the PU comprises selecting the first stage motion vector from a merge mode candidate list for the PU.
4. The method of claim 1 , further comprising generating an advanced TMVP candidate in a merge mode candidate list for the PU, wherein the advanced TMVP candidate indicates performance of the advanced TMVP mode to predict the PU.
5. The method of claim 1 , further comprising, based on the at least one motion vector of the second stage motion information for one of the sub-PUs being unavailable, using a representative motion vector for the one of the sub-PUs.
6. The method of claim 1 , further comprising determining motion refinement data for the PU predicted using the advanced TMVP mode.
7. The method of claim 6 , wherein determining the motion refinement data comprises determining a single motion vector difference for the PU that is applicable to the second stage motion information for each of the sub-PUs of the PU.
8. The method of claim 6 , wherein determining the motion refinement data comprises determining a different motion vector difference for the second stage motion information for each of the sub-PUs.
9. The method of claim 6 , further comprising generating an advanced TMVP candidate in an advanced motion vector prediction (AMVP) mode candidate list for the PU, wherein the advanced TMVP candidate indicates performance of the advanced TMVP mode using the motion refinement data to predict the current PU.
10. The method of claim 1 , further comprising:
generating a predictive block for each of the sub-PUs of the PU based on the second motion information;
generating a residual block based on the video block and the predictive block for each of the sub-PUs; and
encoding the residual block and an indicator of at least the first motion stage motion vector for the PU in a video bitstream.
11. The method of claim 1 , further comprising:
decoding a residual block and an indicator of at least the first stage motion vector for the PU from a received video bitstream;
generating a predictive block for each of the sub-PUs of the PU based on the second motion information; and
generating a reconstructed version of the video block based on the residual block and the predictive block for each of the sub-PUs.
12. A video processing device comprising:
a memory configured to store video data; and
one or more processors in communication with the memory and configured to:
determine a first stage motion vector for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU as a first stage of an advanced temporal motion vector prediction (TMVP) mode, wherein the first stage motion vector identifies a block of a reference picture corresponding to the PU,
partition the PU into two or more sub-PUs,
determine second stage motion information for each of the sub-PUs from the block of the reference picture identified by the first stage motion vector as a second stage of the advanced TMVP mode, wherein the second stage motion information for each of the sub-PUs includes at least one motion vector and an associated reference index, and
perform motion compensation for each of the sub-PUs separately based on the second stage motion information for each of the sub-PUs.
13. The device of claim 12 , wherein the one or more processors are configured to derive the first stage motion vector from one or more of spatial neighboring blocks or temporal neighboring blocks of the PU.
14. The device of claim 12 , wherein the one or more processors at configured to select the first stage motion vector from a merge mode candidate list for the PU.
15. The device of claim 12 , wherein the one or more processors are configured to generate an advanced TMVP candidate in a merge mode candidate list for the PU, wherein the advanced TMVP candidate indicates performance of the advanced TMVP mode to predict the PU.
16. The device of claim 12 , wherein, based on the at least one motion vector of the second stage motion information for one of the sub-PUs being unavailable, the one or more processors are configured to use a representative motion vector for the one of the sub-PUs.
17. The device of claim 12 , wherein the one or more processors are configured to determine motion refinement data for the PU predicted using the advanced TMVP mode.
18. The device of claim 17 , wherein the one or more processors are configured to determine a single motion vector difference for the PU that is applicable to the second stage motion information for each of the sub-PUs of the PU.
19. The device of claim 17 , wherein the one or more processors are configured to determine a different motion vector difference for the second stage motion information for each of the sub-PUs.
20. The device of claim 17 , wherein the one or more processors are configured to generate an advanced TMVP candidate in an advanced motion vector prediction (AMVP) mode candidate list for the PU, wherein the advanced TMVP candidate indicates performance of the advanced TMVP mode using the motion refinement data to predict the current PU.
21. The device of claim 12 , wherein the video processing device comprises a video encoding device, and wherein the one or more processors are configured to:
generate a predictive block for each of the sub-PUs of the PU based on the second motion information;
generate a residual block based on the video block and the predictive block for each of the sub-PUs; and
encode the residual block and an indicator of at least the first motion stage motion vector for the PU in a video bitstream.
22. The device of claim 12 , wherein the video processing device comprises a video decoding device, and wherein the one or more processors are configured to:
decode a residual block and an indicator of at least the first stage motion vector for the PU from a received video bitstream;
generate a predictive block for each of the sub-PUs of the PU based on the second motion information; and
generate a reconstructed version of the video block based on the residual block and the predictive block for each of the sub-PUs.
23. A video processing device comprising:
means for determining a first stage motion vector for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU as a first stage of an advanced temporal motion vector prediction (TMVP) mode, wherein the first stage motion vector identifies a block of a reference picture corresponding to the PU;
means for partitioning the PU into two or more sub-PUs;
means for determining second stage motion information for each of the sub-PUs from the block of the reference picture identified by the first stage motion vector as a second stage of the advanced TMVP mode, wherein the second stage motion information for each of the sub-PUs includes at least one motion vector and an associated reference index; and
means for performing motion compensation for each of the sub-PUs separately based on the second stage motion information for each of the sub-PUs.
24. A computer-readable storage medium storing instructions for processing video data that, when executed, cause one or more processors to:
determine a first stage motion vector for a prediction unit (PU) of a coding unit (CU) of a video block from neighboring blocks of the PU as a first stage of an advanced temporal motion vector prediction (TMVP) mode, wherein the first stage motion vector identifies a block of a reference picture corresponding to the PU;
partition the PU into two or more sub-PUs;
determine second stage motion information for each of the sub-PUs from the block of the reference picture identified by the first stage motion vector as a second stage of the advanced TMVP mode, wherein the second stage motion information for each of the sub-PUs includes at least one motion vector and an associated reference index; and
perform motion compensation for each of the sub-PUs separately based on the second stage motion information for each of the sub-PUs.