360-degree video coding using geometry projection
Processing video data may include capturing the video data with multiple cameras and stitching the video data together to obtain a 360-degree video. A frame-packed picture may be provided based on the captured and stitched video data. A current sample location may be identified in the frame-packed picture. Whether a neighboring sample location is located outside of a content boundary of the frame-packed picture may be determined. When the neighboring sample location is located outside of the content boundary, a padding sample location may be derived based on at least one circular characteristic of the 360-degree video content and the projection geometry. The 360-degree video content may be processed based on the padding sample location.
1 . A method of video decoding comprising:
obtaining a current block location in a video content, wherein the current block location is associated with a current block of the video content, the video content comprising a plurality of faces;
obtaining a spatial neighboring block location, wherein the current block is configured to be decoded based on the spatial neighboring block location;
determining that the spatial neighboring block location is located outside of a content boundary of the video content;
based on the determination that the spatial neighboring block location is located outside of the content boundary of the video content, obtaining a mapped block location associated with the spatial neighboring block location including:
projecting the spatial neighboring block location along one direction being a perpendicular direction or a diagonal direction to a diagonal line extending out of a corner of a face of the plurality of faces to obtain an intermediate location; and
projecting the intermediate location along the one direction to a second content boundary to obtain the mapped block location associated with the spatial neighboring block location;
obtaining a prediction mode attribute associated with the mapped block location, wherein the prediction mode attribute includes an intra mode of a block at the mapped block location; and
based on the prediction mode attribute, decoding the current block.
2 . The method of claim 1 , wherein the method further comprises:
reconstructing the current block based on the intra mode of the block at the mapped block location.
3 . The method of claim 1 , wherein the video content comprises a plurality of faces, and obtaining the mapped block location comprises:
calculating a 3D position associated with the spatial neighboring block location, wherein the spatial neighboring block location is associated with a first face;
based on the calculated 3D position associated with the spatial neighboring block location, identifying a second face that is associated with the mapped block location; and
applying a geometry projection with the 3D position of the spatial neighboring block location to derive a 2D planar position of the spatial neighboring block location in the second face as the mapped block location.
4 . The method of claim 3 , wherein the video content comprises a plurality of faces associated with a first geometry projection, and wherein the method further comprises:
converting a coordinate associated with the first geometry projection into an intermediate coordinate, wherein the intermediate coordinate is associated with a second geometry projection, wherein the 3D position of the spatial neighboring block location is obtained in the intermediate coordinate, and the 2D planar position of the spatial neighboring block location in the second face is obtained in the intermediate coordinate; and
converting the derived 2D planar position of spatial neighboring block location associated with the second geometry projection back to the coordinate associated with the first geometry projection.
5 . The method of claim 1 , wherein the video content comprises a plurality of faces, wherein the spatial neighboring block location is located in a first face, wherein the mapped block location is located in a second face, and wherein the second face differs from the first face.
6 . A non-transitory computer readable medium storing instructions, which, when executed by one or more processors, cause the one or more processors to perform the method of claim 1 .
7 . An apparatus for video decoding comprising:
a processor configured to:
obtain a current block location in a video content, wherein the current block location is associated with a current block of the video content, the video content comprising a plurality of faces;
obtain a spatial neighboring block location, wherein the current block is configured to be decoded based on the spatial neighboring block location;
determine that the spatial neighboring block location is located outside of a content boundary of the video content;
based on the determination that the spatial neighboring block location is located outside of the content boundary of the video content, obtain a mapped block location associated with the spatial neighboring block location including:
project the spatial neighboring block location along one direction being a perpendicular direction or a diagonal direction to a diagonal line extending out of a corner of a face of the plurality of faces to obtain an intermediate location; and
project the intermediate location along the one direction to a second content boundary to obtain the mapped block location associated with the spatial neighboring block location;
obtain a prediction mode attribute associated with the mapped block location, wherein the prediction mode attribute includes an intra mode of a block at the mapped block location; and
based on the prediction mode attribute, decode the current block.
8 . The apparatus of claim 7 , wherein the processor is further configured to:
reconstruct the current block based on the intra mode of the block at the mapped block location.
9 . The apparatus of claim 7 , wherein the video content comprises a plurality of faces and to obtain the mapped block location comprises the processor being configured to:
calculate a 3D position associated with the spatial neighboring block location, wherein the spatial neighboring block location is associated with a first face;
based on the calculated 3D position associated with the spatial neighboring block location, identify a second face that is associated with the mapped block location; and
apply a geometry projection with the 3D position of the spatial neighboring block location to derive a 2D planar position of the spatial neighboring block location in the second face as the mapped block location.
10 . The apparatus of claim 9 , wherein the video content comprises a plurality of faces associated with a first geometry projection, and wherein the processor is further configured to:
convert a coordinate associated with the first geometry projection into an intermediate coordinate, wherein the intermediate coordinate is associated with a second geometry projection, wherein the 3D position of the spatial neighboring block location is obtained in the intermediate coordinate, and the 2D planar position of the spatial neighboring block location in the second face is obtained in the intermediate coordinate; and
convert the derived 2D planar position of the spatial neighboring block location associated with the second geometry projection back to the coordinate associated with the first geometry projection.
11 . A method of video encoding comprising:
obtaining a current block location in a video content, wherein the current block location is associated with a current block of the video content, the video content comprising a plurality of faces;
obtaining a spatial neighboring block location, wherein the current block is configured to be encoded based on the spatial neighboring block location;
determining that the spatial neighboring block location is located outside of a content boundary of the video content;
based on the determination that the spatial neighboring block location is located outside of the content boundary of the video content, obtaining a mapped block location associated with the spatial neighboring block location including:
projecting the spatial neighboring block location along one direction being a perpendicular direction or a diagonal direction to a diagonal line extending out of a corner of a face of the plurality of faces to obtain an intermediate location; and
projecting the intermediate location along the one direction to a second content boundary to obtain the mapped block location associated with the spatial neighboring block location;
obtaining a prediction mode attribute associated with the mapped block location, wherein the prediction mode attribute includes an intra mode of a block at the mapped block location; and
encoding the current block based on the obtained prediction mode attribute associated with the mapped block location.
12 . A non-transitory computer readable medium storing instructions, which, when executed by one or more processors, cause the one or more processors to perform the method of claim 11 .
13 . The apparatus of claim 7 , wherein the video content comprises a plurality of faces, wherein the spatial neighboring block location is located in a first face, wherein the mapped block location is located in a second face, and wherein the second face differs from the first face, wherein the content boundary comprises at least one of a frame packed picture boundary or a face boundary, and wherein the video content comprises a 360-degree video content, and the mapped block location is obtained based on a circular characteristic of the 360-degree video content.
14 . A apparatus for video encoding comprising:
a processor configured to:
obtain a current block location in a video content, wherein the current block location is associated with a current block of the video content, the video content comprising a plurality of faces;
obtain a spatial neighboring block location, wherein the current block is configured to be encoded based on the spatial neighboring block location;
determine that the spatial neighboring block location is located outside of a content boundary of the video content;
based on the determination that the spatial neighboring block location is located outside of the content boundary of the video content, obtain a mapped block location associated with the spatial neighboring block location including:
project the spatial neighboring block location along one direction being a perpendicular direction or a diagonal direction to a diagonal line extending out of a corner of a face of the plurality of faces to obtain an intermediate location; and
project the intermediate location along the one direction to a second content boundary to obtain the mapped block location associated with the spatial neighboring block location;
obtain a prediction mode attribute associated with the mapped block location, wherein the prediction mode attribute includes an intra mode of a block at the mapped block location; and
encode the current block based on the obtained prediction mode attribute associated with the mapped block location.
15 . The apparatus of claim 14 , wherein the video content comprises a plurality of faces.