Fisheye rendering with lens distortion correction for 360-degree video

1. A method of processing 360-degree video data, comprising:

obtaining two-dimensional video data captured by an omnidirectional camera, wherein the two-dimensional video data includes an image of a scene, and wherein, in the image, the scene has been warped into a circular region of the image;

determining, for pixel locations in the circular region of the image, corresponding locations on a three-dimensional hemispherical representation of the image, wherein determining the corresponding locations includes applying a function that projects a point in the image, represented by a two-dimensional coordinate defined by a first angle in an image plane of the image and a distance from a center of the circular region of the image, to a corresponding point on the three-dimensional hemispherical representation, represented by a three-dimensional coordinate defined by a second angle in the image plane and a third angle in an orthogonal plane from the image plane;

mapping, using the two-dimensional coordinate and the three-dimensional coordinate, a pixel in the circular region of the image to the three-dimensional hemispherical representation of the image, wherein the point in the image provides the pixel to map to the point on the three-dimensional hemispherical representation; and

outputting the three-dimensional hemispherical representation for display in a 360-degree video presentation,

wherein the function is a polynomial function, of degree at least 2, in the distance from the center of the circular region of the image, and wherein the function applies a parameter of the omnidirectional camera to determine the three-dimensional coordinate.

2. The method of claim 1 , wherein the warping is non-linear, and further comprising:

determining the two-dimensional coordinate using an adjusted radial value, wherein the adjusted radial value corresponds to the non-linearity of the warping.

3. The method of claim 1 , wherein the image includes a field of view that is greater than 180 degrees, and further comprising:

adjusting the two-dimensional coordinate using a radial value that has been scaled according to the field of view; and

using the adjusted two-dimensional coordinate in the mapping.

4. The method of claim 1 , wherein the image includes a field of view of at least 180 degrees.

5. The method of claim 1 , wherein the omnidirectional camera includes a fisheye lens.

6. The method of claim 1 , wherein the 360-degree video data is obtained from an encoded bitstream, wherein the encoded bitstream includes the parameter of the omnidirectional camera.

7. The method of claim 6 , wherein the parameter of the omnidirectional camera is encoded in a supplemental enhancement information (SEI) message in the encoded bitstream.

8. The method of claim 6 , wherein the encoded bitstream is included in a file formatted according to an ISO base media file format, and wherein the parameter of the omnidirectional camera is encoded in a structure in the file.

9. The method of claim 6 , wherein the parameter of the omnidirectional camera is a polynomial order, a polynomial coefficient, a polynomial scaling parameter, or a mapping scaling parameter.

10. An apparatus for processing encoded video, comprising:

a memory configured to store two-dimensional video data captured by an omnidirectional camera; and

a processor implemented in circuitry and configured to:

obtain the two-dimensional video data, wherein the two-dimensional video data includes an image of a scene, and wherein, in the image, the scene has been warped into a circular region of the image;

determine, for pixel locations in the circular region of the image, corresponding locations on a three-dimensional hemispherical representation of the image, wherein determining the corresponding locations includes applying a function that projects a point in the image, represented by a two-dimensional coordinate defined by a first angle in an image plane of the image and a distance from a center of the circular region of the image, to a corresponding point on the three-dimensional hemispherical representation, represented by a three-dimensional coordinate defined by a second angle in the image plane and a third angle in an orthogonal plane from the image plane;

map, using the two-dimensional coordinate and the three-dimensional coordinate, a pixel in the circular region of the image to the three-dimensional hemispherical representation of the image, wherein the point in the image provides the pixel to map to the point on the three-dimensional hemispherical representation; and

output the three-dimensional hemispherical representation for display in a 360-degree video presentation,

wherein the function is a polynomial function, of degree at least 2, in the distance from the center of the circular region of the image, and wherein the function applies a parameter of the omnidirectional camera to determine the three-dimensional coordinate.

11. The apparatus of claim 10 , wherein the warping is non-linear, and wherein the processor is further configured to:

determine the two-dimensional coordinate using an adjusted radial value, wherein the adjusted radial value corresponds to the non-linearity of the warping.

12. The apparatus of claim 10 , wherein the image includes a field of view that is greater than 180 degrees, and wherein the processor is further configured to:

adjust the two-dimensional coordinate using a radial value that has been scaled according to the field of view; and

use the adjusted two-dimensional coordinate in the mapping.

13. The apparatus of claim 10 , wherein the image includes a field of view of at least 180 degrees.

14. The apparatus of claim 10 , wherein the omnidirectional camera includes a fisheye lens.

15. The apparatus of claim 10 , wherein the 360-degree video data is obtained from an encoded bitstream, wherein the encoded bitstream includes the parameter of the omnidirectional camera.

16. The apparatus of claim 15 , wherein the parameter of the omnidirectional camera is encoded in a supplemental enhancement information (SEI) message in the encoded bitstream.

17. The apparatus of claim 15 , wherein the encoded bitstream is included in a file formatted according to an ISO base media file format, and wherein the parameter of the omnidirectional camera is encoded in a structure in the file.

18. The apparatus of claim 15 , wherein the parameter of the omnidirectional camera is a polynomial order, a polynomial coefficient, a polynomial scaling parameter, or a mapping scaling parameter.

19. The apparatus of claim 10 , further comprising:

a mobile device, wherein the mobile device includes the omnidirectional camera.

20. An apparatus for decoding video, comprising:

a memory configured to store two-dimensional video data captured by an omnidirectional camera; and

a processor implemented in circuitry, configured to:

obtain the two-dimensional video data, wherein the two-dimensional video data includes an image of a scene, and wherein, in the image, the scene has been warped into a circular region of the image;

determine, for pixel locations in the circular region of the image, corresponding locations on a three-dimensional hemispherical representation of the image, wherein determining the corresponding locations includes applying a function that projects a point in the image, represented by a two-dimensional coordinate defined by a first angle in an image plane of the image and a distance from a center of the circular region of the image, to a corresponding point on the three-dimensional hemispherical representation, represented by a three-dimensional coordinate defined by a second angle in the image plane and a third angle in an orthogonal plane from the image plane;

map, using the two-dimensional coordinate and the three-dimensional coordinate, a pixel in the circular region of the image to the three-dimensional hemispherical representation of the image, wherein the point in the image provides the pixel to map to the point on the three-dimensional hemispherical representation; and

output the three-dimensional hemispherical representation for display in a 360-degree video presentation,

wherein the function is a polynomial function, of degree at least 2, in the distance from the center of the circular region of the image, and wherein the function applies a parameter of the omnidirectional camera to determine the three-dimensional coordinate.

21. The apparatus of claim 20 , wherein the warping is non-linear, and wherein the processor is further configured to:

determine the two-dimensional coordinate using an adjusted radial value, wherein the adjusted radial value corresponds to the non-linearity of the warping.

22. The apparatus of claim 20 , wherein the image includes a field of view that is greater than 180 degrees, and wherein the processor is further configured to:

adjust the two-dimensional coordinate using a radial that has been scaled according to the field of view; and

use the adjusted two-dimensional coordinate in the mapping.

23. The apparatus of claim 20 , wherein the image includes a field of view of at least 180 degrees.

24. The apparatus of claim 20 , wherein the omnidirectional camera includes a fisheye lens.

25. The apparatus of claim 20 , wherein the 360-degree video data is obtained from an encoded bitstream, wherein the encoded bitstream includes the parameter of the omnidirectional camera.

26. The apparatus of claim 25 , wherein the parameter of the omnidirectional camera is encoded in a supplemental enhancement information (SEI) message in the encoded bitstream.

27. The apparatus of claim 25 , wherein the encoded bitstream is included in a file formatted according to an ISO base media file format, and wherein the parameter of the omnidirectional camera is encoded in a structure in the file.

28. The apparatus of claim 25 , wherein the parameter of the omnidirectional camera is a polynomial order, a polynomial coefficient, a polynomial scaling parameter, or a mapping scaling parameter.

29. The apparatus of claim 20 , further comprising:

a mobile device, wherein the mobile device includes the omnidirectional camera.

30. The apparatus of claim 20 , further comprising:

a mobile device, wherein the mobile device includes a display for displaying the 360-degree video presentation.

31. A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to:

obtain two-dimensional video data captured by an omnidirectional camera, wherein the two-dimensional video data includes an image of a scene, and wherein, in the image, the scene has been warped into a circular region of the image;

determine, for pixel locations in the circular region of the image, corresponding locations on a three-dimensional hemispherical representation of the image, wherein determining the corresponding locations includes applying a function that projects a point in the image, represented by a two-dimensional coordinate defined by a first angle in an image plane of the image and a distance from a center of the circular region of the image, to a corresponding point on the three-dimensional hemispherical representation, represented by a three-dimensional coordinate defined by a second angle in the image plane and a third angle in an orthogonal plane from the image plane;

map, using the two-dimensional coordinate and the three-dimensional coordinate, a pixel in the circular region of the image to the three-dimensional hemispherical representation of the image, wherein the point in the image provides the pixel to map to the point on the three-dimensional hemispherical representation; and

output the three-dimensional hemispherical representation for display in a 360-degree video presentation,

wherein the function is a polynomial function, of degree at least 2, in the distance from the center of the circular region of the image, and wherein the function applies a parameter of the omnidirectional camera to determine the three-dimensional coordinate.

32. An apparatus, comprising:

means for obtaining two-dimensional video data, wherein the two-dimensional video data includes an image of a scene, and wherein, in the image, the scene has been warped into a circular region of the image;

means for determining, for pixel locations in the circular region of the image, corresponding locations on a three-dimensional hemispherical representation of the image, wherein determining the corresponding locations includes applying a function that projects a point in the image, represented by a two-dimensional coordinate defined by a first angle in an image plane of the image and a distance from a center of the circular region of the image, to a corresponding point on the three-dimensional hemispherical representation, represented by a three-dimensional coordinate defined by a second angle in the image plane and a third angle in an orthogonal plane from the image plane;

means for mapping, using the two-dimensional coordinate and the three-dimensional coordinate, a pixel in the circular region of the image to the three-dimensional hemispherical representation of the image, wherein the point in the image provides the pixel to map to the point on the three-dimensional hemispherical representation; and

means for outputting the three-dimensional hemispherical representation for display in a 360-degree video presentation,

wherein the function is a polynomial function, of degree at least 2, in the distance from the center of the circular region of the image, and wherein the function applies a parameter of the omnidirectional camera to determine the three-dimensional coordinate.