Predictive coding for 360-degree video based on geometry padding
A video coding system (e.g., an encoder and/or a decoder) may perform face-based sub-block motion compensation for 360-degree video to predict samples (e.g., of a sub-block). The video coding system may receive a 360-degree video content. The 360-degree video content may include a current block. The current block may include a plurality of sub-blocks. The system may determine whether a sub-block mode is used for the current block. The system may predict a sample in the current block based on the sub-block level face association. For a first sub-block in the current block, the system may identify a first location of the first sub-block. The system may associate the first sub-block with a first face based on the identified first location of the first sub-block. The system may predict a first sample in the first sub-block based on the first face that is associated with the first sub-block.
1. A method of video decoding comprising:
obtaining a 360-degree video content comprising a current block;
determining that a sub-block mode is used for the current block, the current block comprising a plurality of sub-blocks;
determining that the current block comprises samples located on a plurality of faces associated with the 360-degree video content; and
predicting at least one sample in the current block based on sub-block level face association;
wherein for a first sub-block in the current block:
identifying a first location of the first sub-block,
associating the first sub-block with a first face based on the identified first location of the first sub-block,
identifying a first reference sample based on the first sub-block being associated with the first face, and
predicting a first sample in the first sub-block based on the first reference sample associated with the first face that is associated with the first sub-block; and
wherein for a second sub-block in the current block:
identifying a second location of the second sub-block,
associating the second sub-block with a second face based on the identified second location of the second sub-block,
identifying a second reference sample based on the second sub-block being associated with the second face, and
predicting a second sample in the second sub-block based on the second reference sample associated with the second face that is associated with the second sub-block.
2. The method of claim 1 , wherein the second face differs from the first face.
3. The method of claim 1 , wherein predicting the first sample associated with the first sub-block further comprising:
identifying the first reference sample associated with the first face using a motion vector; and
predicting the first sample in the first sub-block based on the identified first reference sample.
4. The method of claim 3 , wherein predicting the first sample in the first sub-block further comprises predicting the first sample using the identified first reference sample.
5. The method of claim 3 , wherein the identified first reference sample is located within the first face or a padded region of the first face.
6. The method of claim 1 , wherein the 360-degree video content comprises the plurality of faces arranged in a frame-packed picture, and the sub-block level face association is performed based on the locations of the plurality of the sub-blocks in the frame-packed picture.
7. The method of claim 1 , wherein predicting at least one sample in the current block based on the sub-block level face association further comprises:
on a condition that the first sub-block is associated with the first face and a second sub-block is associated with a second face and the first face differs from the second face, predicting the first sample using the first reference sample that is associated with the first face and predicting a second sample using the second reference sample that is associated with the second face.
8. The method of claim 1 , wherein the sub-block mode comprises at least one of an advanced temporal motion vector prediction (ATMVP) mode, a spatial-temporal motion vector prediction (STMVP) mode, a frame-rate up conversion (FRUC) mode, or an affine mode.
9. The method of claim 1 , further comprising:
performing geometry padding on the plurality of faces and the first face comprises a padded region associated with the first face.
10. An apparatus for video decoding comprising:
a processor configured to:
obtain a 360-degree video content comprising a current block;
determine that a sub-block mode is used for the current block, the current block comprising a plurality of sub-blocks;
determine that the current block comprises samples located on a plurality of faces associated with the 360-degree video content; and
predict at least one sample in the current block based on sub-block level face association;
wherein for a first sub-block in the current block:
identify a first location of the first sub-block,
associate the first sub-block with a first face based on the identified first location of the first sub-block,
identify a first reference sample based on the first sub-block being associated with the first face, and
predict a first sample in the first sub-block based on the first reference sample associated with the first face that is associated with the first sub-block; and
wherein for a second sub-block in the current block:
identifying a second location of the second sub-block,
associating the second sub-block with a second face based on the identified second location of the second sub-block,
identifying a second reference sample based on the second sub-block being associated with the second face, and
predicting a second sample in the second sub-block based on the second reference sample associated with the second face that is associated with the second sub-block.
11. The apparatus of claim 10 , wherein the second face differs from the first face.
12. The apparatus of claim 10 , wherein the processor for predicting the first sample associated with the first sub-block is further configured to:
identify the first reference sample associated with the first face using a motion vector; and
predict the first sample in the first sub-block based on the identified first reference sample.
13. The apparatus of claim 12 , wherein the processor for predicting the first sample in the first sub-block is further configured to predict the first sample using the identified first reference sample.
14. The apparatus of claim 12 , wherein the identified first reference sample is located within the first face or a padded region of the first face.
15. The apparatus of claim 10 , wherein the 360-degree video content comprises the plurality of faces arranged in a frame-packed picture, and the sub-block level face association is performed based on the locations of the plurality of the sub-blocks in the frame-packed picture.
16. The apparatus of claim 10 , wherein the processor for predicting at least one sample in the current block based on the sub-block level face association is further configured to:
on a condition that the first sub-block is associated with the first face and a second sub-block is associated with a second face and the first face differs from the second face, predict the first sample using the first reference sample that is associated with the first face and predict a second sample using the second reference sample that is associated with the second face.
17. The apparatus of claim 10 , wherein the sub-block mode comprises at least one of an advanced temporal motion vector prediction (ATMVP) mode, a spatial-temporal motion vector prediction (STMVP) mode, a frame-rate up conversion (FRUC) mode, or an affine mode.
18. The apparatus of claim 10 , wherein the processor is further configured to:
perform geometry padding on the plurality of faces and the first face comprises a padded region associated with the first face.