360-degree video coding using geometry projection
Processing video data may include capturing the video data with multiple cameras and stitching the video data together to obtain a 360-degree video. A frame-packed picture may be provided based on the captured and stitched video data. A current sample location may be identified in the frame-packed picture. Whether a neighboring sample location is located outside of a content boundary of the frame-packed picture may be determined. When the neighboring sample location is located outside of the content boundary, a padding sample location may be derived based on at least one circular characteristic of the 360-degree video content and the projection geometry. The 360-degree video content may be processed based on the padding sample location.
1. A method comprising:
obtaining a current sample location in a frame-packed picture, wherein the frame-packed picture comprises a plurality of faces associated with a first projection geometry, wherein the current sample location is located in a first face associated with the frame-packed picture, and wherein the current sample location is associated with a current sample of a video content;
obtaining a first prediction sample location, wherein the current sample is to be predicted based on the first prediction sample location;
determining that the first prediction sample location is located outside of a boundary of the first face;
based on the determination that the first prediction sample location is located outside of the boundary of the first face, obtaining a 3D position of the first prediction sample location, wherein the 3D position of the first prediction sample location is associated with the first face;
based on the 3D position of the first prediction sample location, obtaining a second face that includes a second prediction sample location;
applying a geometry projection to the 3D position of the first prediction sample location to derive a 2D planar position of the first prediction sample location in the second face as the second prediction sample location; and
predicting the current sample associated with the video content based on the second prediction sample location.
2. The method of claim 1 , comprising:
determining that the current sample is intra predicted; and
based on the determination that the current sample is intra predicted and based on the determination that the first prediction sample location is located outside of the first face, obtaining the first prediction sample location based on an intra prediction mode associated with the current sample.
3. The method of claim 1 , comprising:
determining that the current sample is inter predicted; and
based on the determination that the current sample is inter predicted and based on the determination that the first prediction sample location is located outside of the boundary of the first face, obtaining the first prediction sample location by applying a motion vector associated with the current sample to the current sample.
4. The method of claim 1 , wherein the first projection geometry comprises at least one of an equirectangular projection, an equal-area projection, a cubemap projection, an octahedron projection, or a cylinder projection.
5. The method of claim 1 , wherein the method further comprises:
converting a coordinate in the first projection geometry into an intermediate coordinate, wherein the intermediate coordinate is associated with a second projection geometry, wherein the 3D position of the first prediction sample location is obtained in the intermediate coordinate, and the 2D planar position of the first prediction sample location in the second face is obtained in the intermediate coordinate; and
converting the derived 2D planar position of the first prediction sample location associated with the second projection geometry back to the coordinate associated with the first projection geometry.
6. An apparatus comprising:
a processor configured to:
obtain a current sample location in a frame-packed picture, wherein the frame-packed picture comprises a plurality of faces associated with a first projection geometry, wherein the current sample location is located in a first face associated with the frame-packed picture, wherein the current sample location is associated with a current sample of a video content;
obtain a first prediction sample location, wherein the current sample is to be predicted based on the first prediction sample location;
determine that the first prediction sample location is outside of a boundary of the first face;
based on the determination that the first prediction sample location is outside of the first face, obtain a 3D position of the first prediction sample location, wherein the 3D position of the first prediction sample location is associated with the first face;
based on the 3D position of the first prediction sample location, obtain a second face that includes a second prediction sample location;
apply a geometry projection with the 3D position of the first prediction sample location to derive a 2D planar position of the first prediction sample location in the second face, wherein the 2D planar position of the first prediction sample location is the second prediction sample location; and
predict the current sample associated with the video content based on the second prediction sample location.
7. The apparatus of claim 6 , wherein the processor is configured to:
determine that the current sample is intra predicted; and
based on the determination that the current sample is intra predicted and based on the determination that the first prediction sample location is located outside of the first face, obtain the first prediction sample location based on an intra prediction mode associated with the current sample.
8. The apparatus of claim 6 , wherein the processor is configured to:
determine that the current sample is inter predicted; and
based on the determination that the current sample is inter predicted and based on the determination that the first prediction sample location is located outside of the boundary of the first face, obtain the first prediction sample location by applying a motion vector associated with the current sample to the current sample.
9. The apparatus of claim 6 , wherein the first projection geometry comprises at least one of an equirectangular projection, an equal-area projection, a cubemap projection, an octahedron projection, or a cylinder projection.
10. The apparatus of claim 6 , wherein the processor is further configured to:
convert a coordinate in the first projection geometry into an intermediate coordinate, wherein the intermediate coordinate is associated with a second projection geometry, wherein the 3D position of the first prediction sample location is obtained in the intermediate coordinate, and the 2D planar position of the first prediction sample location in the second face is obtained in the intermediate coordinate; and
convert the derived 2D planar position of the first prediction sample location associated with the second projection geometry back to the coordinate associated with the first projection geometry.