360-degree video coding using geometry projection
Processing video data may include capturing the video data with multiple cameras and stitching the video data together to obtain a 360-degree video. A frame-packed picture may be provided based on the captured and stitched video data. A current sample location may be identified in the frame-packed picture. Whether a neighboring sample location is located outside of a content boundary of the frame-packed picture may be determined. When the neighboring sample location is located outside of the content boundary, a padding sample location may be derived based on at least one circular characteristic of the 360-degree video content and the projection geometry. The 360-degree video content may be processed based on the padding sample location.
1. A method comprising:
identifying a current sample location in a frame-packed picture associated with a first projection geometry, the current sample location being associated with a current sample of a 360-degree video content, wherein the current sample is to be predicted using a reference sample;
determining that a reference sample location associated with the reference sample for predicting the current sample is located outside of a content boundary of the frame-packed picture;
based on the determination that the reference sample location associated with the reference sample for predicting the current sample is located outside of the content boundary of the frame-packed picture, deriving a padding sample location for predicting the current sample based on at least one circular characteristic of the 360-degree video content and the first projection geometry, the padding sample location being associated with a padding sample; and
predicting the current sample associated with the 360-degree video content based on the padding sample.
2. The method of claim 1 , wherein determining that the reference sample location is located outside of the content boundary of the frame-packed picture comprises:
on a condition that the current sample is intra predicted, identifying the reference sample location based on an intra prediction mode associated with the current sample.
3. The method of claim 1 , wherein determining that the reference sample location is located outside of the content boundary of the frame-packed picture comprises:
on a condition that the current sample is inter predicted, identifying the reference sample location by applying a motion vector associated with the current sample to the current sample.
4. The method of claim 1 , wherein the content boundary comprises at least one of a frame packed picture boundary or a face boundary.
5. The method of claim 1 , wherein the first projection geometry comprises at least one of an equirectangular projection, an equal-area projection, a cubemap projection, an octahedron projection, or a cylinder projection.
6. The method of claim 1 , wherein the frame-packed picture comprises a plurality of faces associated with the first projection geometry, wherein the content boundary defines a boundary between faces, and deriving the padding sample location comprises:
identifying a first face that includes the current sample location;
calculating a 3D position of the reference sample location associated with the first face;
identifying a second face that includes the padding sample location based on the 3D position of the reference sample location; and
applying a geometry projection with the 3D position of the reference sample location to derive a 2D planar position of the reference sample location in the second face as the padding sample location.
7. The method of claim 6 , wherein deriving the padding sample location further comprises:
converting a coordinate in the first projection geometry into an intermediate coordinate associated with a second projection geometry, wherein the 3D position of the reference sample location is calculated in the intermediate coordinate, and the 2D planar position of the reference sample location in the second face is identified in the intermediate coordinate; and
converting the derived 2D planar position of the reference sample location associated with the second projection geometry back to the coordinate associated with the first projection geometry.
8. The method of claim 1 , wherein:
when the current sample location is located near a rightmost content boundary of the frame-packed picture and the determined reference sample location is located outside of the frame-packed picture beyond the right content boundary, the padding sample location is located near a leftmost content boundary of the frame-packed picture that is circularly connected to the rightmost content boundary of the frame-packed picture;
when the current sample location is located near the leftmost content boundary of the frame-packed picture and the determined reference sample location is located outside of the frame-packed picture beyond the left content boundary, the padding sample location is located near the rightmost content boundary of the frame-packed picture that is circularly connected to the leftmost content boundary of the frame-packed picture;
when the current sample location is located near a first area of a topmost content boundary of the frame-packed picture and the determined reference sample location is located outside of the frame-packed picture beyond the first area of the topmost content boundary, the padding sample location is located near a second area of the topmost content boundary that is located in a circularly opposite side of the first area of the topmost content boundary; and
when the current sample location is located near a first area of a bottommost content boundary of the frame-packed picture and the determined reference sample location is located outside of the frame-packed picture beyond the first area of the bottommost content boundary, the padding sample location is located near a second area of the bottommost content boundary that is located in a circularly opposite side of the first area of the bottommost content boundary.
9. A device comprising:
a processor configured to:
identify a current sample location in a frame-packed picture associated with a first projection geometry, the current sample location being associated with a current sample of a 360-degree video content, wherein the current sample is to be predicted using a reference sample;
determine that a reference sample location associated with the reference sample for predicting the current sample is located outside of a content boundary of the frame-packed picture;
based on the determination that the reference sample location associated with the reference sample for predicting the current sample is outside of the content boundary of the frame-packed picture, derive a padding sample location for predicting the current sample based on at least one circular characteristic of the 360-degree video content and the first projection geometry, the padding sample location being associated with a padding sample; and
predict the current sample associated with the 360-degree video content based on the padding sample.
10. The device of claim 9 , wherein the processor configured to determine that the reference sample location is located outside of the content boundary of the frame-packed picture comprises:
on a condition that the current sample is intra predicted, identify the reference sample location based on an intra prediction mode associated with the current sample.
11. The device of claim 9 , wherein the processor configured to determine that the reference sample location is located outside of the content boundary of the frame-packed picture comprises:
on a condition that the current sample is inter predicted, identify the reference sample location by applying a motion vector associated with the current sample to the current sample.
12. The device of claim 9 , wherein the content boundary comprises at least one of a frame packed picture boundary or a face boundary, and wherein the first projection geometry comprises at least one of an equirectangular projection, an equal-area projection, a cubemap projection, an octahedron projection, or a cylinder projection.
13. The device of claim 9 , wherein the frame-packed picture comprises a plurality of faces associated with the first projection geometry, and wherein the content boundary defines a boundary between faces, and the processor configured to derive the padding sample location comprises:
identify a first face that includes the current sample location;
calculate a 3D position of the reference sample location associated with the first face;
identify a second face that includes the padding sample location based on the 3D position of the reference sample location; and
apply a geometry projection with the 3D position of the reference sample location to derive a 2D planar position of the reference sample location in the second face as the padding sample location.
14. The device of claim 13 , wherein the processor is further configured to derive the padding sample location comprises:
convert a coordinate in the first projection geometry into an intermediate coordinate associated with a second projection geometry, wherein the 3D position of the reference sample location is calculated in the intermediate coordinate, and the 2D planar position of the reference sample location in a second face is identified in the intermediate coordinate; and
convert the derived 2D planar position of the reference sample location associated with the second projection geometry back to the coordinate associated with the first projection geometry.
15. The device of claim 9 , wherein:
when the current sample location is located near a rightmost content boundary of the frame-packed picture and the determined reference sample location is located outside of the frame-packed picture beyond the right content boundary, the padding sample location is located near a leftmost content boundary of the frame-packed picture that is circularly connected to the rightmost content boundary of the frame-packed picture;
when the current sample location is located near the leftmost content boundary of the frame-packed picture and the determined reference sample location is located outside of the frame-packed picture beyond the left content boundary, the padding sample location is located near the rightmost content boundary of the frame-packed picture that is circularly connected to the leftmost content boundary of the frame-packed picture;
when the current sample location is located near a first area of a topmost content boundary of the frame-packed picture and the determined reference sample location is located outside of the frame-packed picture beyond the first area of the topmost content boundary, the padding sample location is located near a second area of the topmost content boundary that is located in a circularly opposite side of the first area of the topmost content boundary; and
when the current sample location is located near a first area of a bottommost content boundary of the frame-packed picture and the determined reference sample location is located outside of the frame-packed picture beyond the first area of the bottommost content boundary, the padding sample location is located near a second area of the bottommost content boundary that is located in a circularly opposite side of the first area of the bottommost content boundary.
16. A method comprising:
obtaining a video signal comprising a frame-packed picture;
identifying a current sample of a 360-degree video content in the frame-packed picture that is associated with a projection geometry;
identifying a padding sample for predicting the current sample based on at least one circular characteristic of the 360-degree video content and the projection geometry;
deriving a reference sample location associated with the padding sample, the reference sample location being located outside of a content boundary of the frame-packed picture; and
including a reference sample location indication for the reference sample location in a bitstream for predicting the current sample associated with the 360-degree video content.
17. The method of claim 16 , wherein the content boundary comprises at least one of a frame packed picture boundary or a face boundary, and wherein the projection geometry comprises at least one of an equirectangular projection, an equal-area projection, a cubemap projection, an octahedron projection, or a cylinder projection.
18. The method of claim 16 , comprising:
on a condition that the current sample is inter predicted, identifying the reference sample location by applying a motion vector associated with the current sample to the current sample.
19. A device comprising:
a processor configured to:
obtain a video signal comprising a frame-packed picture;
identify a current sample a 360-degree video content in the frame-packed picture that is associated with a projection geometry;
identify a padding sample for predicting the current sample based on at least one circular characteristic of the 360-degree video content and the projection geometry;
derive a reference sample location associated with the padding sample, the reference sample location being outside of a content boundary of the frame-packed picture; and
include a reference sample location indication for the reference sample location in a bitstream for predicting the current sample associated with the 360-degree video content.
20. The device of claim 19 , wherein the frame-packed picture comprises a plurality of faces associated with the projection geometry, and wherein the content boundary defines a boundary between faces.