Method and apparatus for omnidirectional video coding and decoding with adaptive intra prediction
In omnidirectional videos, a straight line in the rendering space may no longer appear as a straight line in the frame to be encoded, due to the projection between the encoding space and the rendering space. To adapt to the geometry of omnidirectional videos, intra prediction is performed along non-straight curves. In particular, references samples in neighboring reconstructed blocks of a current block are copied along non-straight curves to form a predicted block for the current block. The non-straight curves may be computed online based on a single intra prediction directional mode for the current block. The parameters of non-straight curves may be based on the mapping used in forming the omnidirectional videos, and the location/size of the current block. The non-straight curves may be obtained using a look-up table or an analytical function that is dependent on the location and size of the current block.
1. A method for encoding video data, comprising:
accessing a directional mode for intra prediction for a first block of a 2D picture, the 2D picture represented in a first format suitable for encoding, wherein the picture is for rendering in a second format which corresponds to a projection from a 3D surface on which the 2D picture is mapped;
determining a look-up table (LUT) from among a plurality of look-up tables (LUTs) based on the directional mode, wherein the plurality of look-up tables respectively correspond to a plurality of directional modes;
determining a plurality of reference samples in reconstructed blocks adjacent to the first block;
accessing a portion of the LUT based on a location of the first block and forming a predicted block for the first block based on the portion of the LUT, wherein a plurality of entries of the portion of the LUT correspond to a plurality of sample locations of the predicted block, wherein the plurality of entries represent a plurality of non-straight curves in the 2D picture along which the reference samples are propagated to the sample locations of the predicted block to form the predicted block, and wherein the non-straight curves in the 2D picture map to straight lines in the projection of the 2D picture;
encoding the directional mode; and
encoding a difference between the first block and the predicted block.
2. The method of claim 1 , wherein a mapping between the first format and the second format is signaled in a bitstream.
3. The method of claim 1 , wherein the accessing of the portion of the LUT is based on the location of the first block in a vertical direction of the 2D picture.
4. The method of claim 1 , wherein the respective entries of the LUT for a first half of the 2D picture are mirrored for a second half of the 2D picture.
5. The method of claim 1 , wherein the accessing of the portion of the LUT is based on a size of the first block and the location of the first block in a vertical direction of the 2D picture.
6. The method of claim 1 , wherein respective values of the plurality of entries indicate whether the reference samples are in the reconstructed blocks left of the first block or above the first block.
7. The method of claim 1 , wherein a mapping between the first format and the second format is signaled in a bitstream.
8. An apparatus for encoding video data, comprising one or more processors configured to:
access a directional mode for intra prediction for a first block of a 2D picture, the 2D picture represented in a first format suitable for encoding, wherein the picture is for rendering in a second format which corresponds to a projection from a 3D surface on which the 2D picture is mapped;
determine a look-up table (LUT) from among a plurality of look-up tables (LUTs) based on the directional mode, wherein the plurality of look-up tables respectively correspond to a plurality of directional modes;
determine a plurality of reference samples in reconstructed blocks adjacent to the first block;
access a portion of the LUT based on a location of the first block and forming a predicted block for the first block based on the portion of the LUT, wherein a plurality of entries of the portion of the LUT correspond to a plurality of sample locations of the predicted block, wherein the plurality of entries represent a plurality of non-straight curves in the 2D picture along which the reference samples are propagated to the sample locations of the predicted block to form the predicted block, and wherein the non-straight curves in the 2D picture map to straight lines in the projection of the 2D picture
encode the directional mode; and
encode a difference between the first block and the predicted block.
9. The apparatus of claim 8 , wherein a mapping between the first format and the second format is signaled in a bitstream.
10. The apparatus of claim 8 , wherein the accessing of the portion of the LUT is based on the location of the first block in a vertical direction of the 2D picture.
11. The apparatus of claim 8 , wherein the respective entries of the LUT for a first half of the 2D picture are mirrored for a second half of the 2D picture.
12. The apparatus of claim 8 , wherein the accessing of the portion of the LUT is based on a size of the first block and the location of the first block in a vertical direction of the 2D picture.
13. The apparatus of claim 8 , wherein respective values of the plurality of entries indicate whether the reference samples are in the reconstructed blocks left of the first block or above the first block.
14. A method for decoding video data, comprising:
accessing a directional mode for intra prediction for a first block of a 2D picture, the 2D picture represented in a first format for decoder output, wherein a decoded version of the picture is for rendering in a second format which corresponds to a projection from a 3D surface on which the 2D picture is mapped;
determining a look-up table (LUT) from among a plurality of look-up tables (LUTs) based on the directional mode, wherein the plurality of look-up tables respectively correspond to a plurality of directional modes;
determining a plurality of references samples in decoded blocks adjacent to the first block;
accessing a portion of the LUT based on a location of the first block and forming a predicted block for the first block based on the portion of the LUT, wherein a plurality of entries of the portion of the LUT correspond to a plurality of sample locations of the predicted block, wherein the plurality of entries represent a plurality of non-straight curves in the 2D picture along which the reference samples are propagated to the sample locations of the predicted block to form the predicted block, and wherein the non-straight curves in the 2D picture map to straight lines in the projection of the 2D picture; and
decoding the first block using the predicted block.
15. The method of claim 14 , wherein the accessing of the portion of the LUT is based on the location of the first block in a vertical direction of the 2D picture.
16. The method of claim 14 , wherein the respective entries of the LUT for a first half of the 2D picture are mirrored for a second half of the 2D picture.
17. The method of claim 14 , wherein the accessing of the portion of the LUT is based on a size of the first block and the location of the first block in a vertical direction of the 2D picture.
18. The method of claim 14 , wherein respective values of the plurality of entries indicate whether the reference samples are in the reconstructed blocks left of the first block or above the first block.
19. An apparatus for decoding video data, comprising one or more processors configured to:
access a directional mode for intra prediction for a first block of a 2D picture, the 2D picture represented in a first format for decoder output, wherein a decoded version of the picture is for rendering in a second format which corresponds to a projection from a 3D surface on which the 2D picture is mapped;
determine a look-up table (LUT) from among a plurality of look-up tables (LUTs) based on the directional mode, wherein the plurality of look-up tables respectively correspond to a plurality of directional modes;
determine a plurality of references samples in decoded blocks adjacent to the first block;
access a portion of the LUT based on a location of the first block and form a predicted block for the first block based on the portion of the LUT, wherein a plurality of entries of the portion of the LUT correspond to a plurality of sample locations of the predicted block, wherein the plurality of entries represent a plurality of non-straight curves in the 2D picture along which the reference samples are propagated to the sample locations of the predicted block to form the predicted block, and wherein the non-straight curves in the 2D picture map to straight lines in the projection of the 2D picture;
decode the first block using the predicted block.
20. The apparatus of claim 19 , wherein a mapping between the first format and the second format is signaled in a bitstream.
21. The apparatus of claim 19 , wherein the accessing of the portion of the LUT is based on the location of the first block in a vertical direction of the 2D picture.
22. The apparatus of claim 19 , wherein the respective entries of the LUT for a first half of the 2D picture are mirrored for a second half of the 2D picture.
23. The apparatus of claim 19 , wherein the accessing of the portion of the LUT is based on a size of the first block and the location of the first block in a vertical direction of the 2D picture.
24. The apparatus of claim 19 , wherein respective values of the plurality of entries indicate whether the reference samples are in the reconstructed blocks left of the first block or above the first block.