Method for processing and delivering virtual reality content to a user
Aspects of the present disclosure include a process for the delivery of virtual reality content that enables a more immersive experience to the user by implementing resolution-defined projections. The resolution-defined projections are two-dimensional and virtual reality content agnostic. According to some aspects, the projections can be used to generate virtual reality content images and video that can be encoded and/or compressed for processing and transmission with lower or without increasing bandwidth requirements, and to be perceived of a higher resolution. Additional aspects include, the adaptive focus via the resolution-defined projections using the user's viewing direction, optimization data from a feedback loop and/or crowdsourcing data.
1. A system for processing and delivering virtual reality content, the system comprising:
one or more computer readable storage devices configured to store a plurality of computer executable instructions; and
one or more hardware computer processors in communication with the one or more computer readable storage devices and configured to execute the plurality of computer executable instructions in order to cause the system to:
create a two-dimensional bijective resolution-defined projection according to a desired resolution distribution over a full sphere of virtual reality image data;
map the virtual reality image data using the two-dimensional bijective resolution-defined projection representing the full sphere of the virtual reality image data;
derive one or more two-dimensional renditions from the virtual reality image data mapped to the two-dimensional bijective resolution-defined projection, wherein each of the one or more two-dimensional renditions represents the full sphere of the virtual reality image data; and
access at least one of the one or more two-dimensional renditions representing the full sphere of the virtual reality image data;
and cause display by a display device an inverse projection of the one or more two-dimensional renditions to a sphere or other three-dimensional shape.
2. The system of claim 1 , wherein the system further caused to:
repeat the creating and the mapping steps over a plurality of desired-resolution distributions to generate a plurality of two-dimensional renditions of the virtual reality content,
wherein the two-dimensional bijective resolution-defined projection is created according to an existing two-dimensional rectangular projection having a separable Jacobian and a separable desired resolution distribution.
3. The system of claim 2 , wherein the system is further caused to:
receive sensor data to determine a user's viewing direction; and
dynamically select at least one of the one or more two-dimensional renditions according to the determined viewing direction of the user.
4. A computer-implemented method for virtual reality content delivery, the method comprising;
creating, by a computer system, a two-dimensional bijective resolution-defined projection according to a desired resolution distribution over a full sphere of virtual reality image data;
mapping, by the computer system, the virtual reality image data using the two-dimensional bijective resolution-defined projection representing the full sphere of the virtual reality image data;
deriving, by the computer system, one or more two-dimensional renditions from the virtual reality image data mapped to the two-dimensional bijective resolution-defined projection, wherein each of the one or more two-dimensional renditions represents the full sphere of the virtual reality image data; and
processing, by the computer system, the one or more two-dimensional renditions representing the full sphere of the virtual reality image data; and
causing displaying by a display device an inverse projection of the one or more two-dimensional renditions to a sphere or other three-dimensional shape,
wherein the computer system comprises a computer processor and an electronic storage medium.
5. The method of claim 4 , wherein the creation of the two-dimensional bijective resolution-defined projection is based on an existing two-dimensional rectangular projection having a separable Jacobain.
6. The method of claim 5 , wherein the creating of the two-dimensional bijective resolution-defined projection is further based on a separable desired resolution distribution.
7. The method of claim 4 , wherein the desired resolution distribution is defined by at least one or more of: a fovea field of view, contrast of the virtual reality image data, movement of the virtual reality image data content, capturing camera arrangement and geometry, capturing camera lenses, display parameters, and crowd sourced data relating to the virtual reality image content.
8. The method of claim 7 , wherein the creating and the mapping are repeated over a plurality of desired-resolution distributions to generate a plurality of two-dimensional renditions of the virtual reality content.
9. The method of claim 8 , additionally comprising: compressing at least one of the plurality one or more two dimensional renditions.
10. The method of claim 8 , wherein at least one of the plurality of two-dimensional renditions is compressed at one or more resolutions and/or encoded at one or more resolutions.
11. The method of claim 8 , wherein at least one of the plurality of two-dimensional renditions is compressed at one or more bitrates and/or encoded at one or more bitrates.
12. The method of claim 8 , additionally comprising:
detecting a user's viewing direction; and
dynamically selecting a subset of the plurality of two-dimensional renditions for the transmitting, wherein the subset is dynamically selected according to the detected viewing direction of the user.
13. A computer-implemented method for virtual reality content delivery, the method comprising:
creating, by a computer system, two or more two-dimensional bijective resolution-defined projections, wherein each of the two or more two-dimensional bijective resolution-defined projections is created according to a desired resolution distribution over a full sphere of virtual reality image data;
mapping, by the computer system, the virtual reality image data using the two or more two-dimensional bijective resolution-defined projections to generate two or more two-dimensional renditions, wherein each of the two or more two-dimensional renditions represents the full sphere of the virtual reality image data; and
accessing, by the computer system, at least one of the two or more two-dimensional renditions representing the full sphere of the virtual reality image data; and
dynamically inversely projecting the at least one of the two or more two-dimensional renditions as immersive video to a user on a display,
wherein the computer system comprises a computer processor and an electronic storage medium.
14. The method of claim 13 , wherein the creation of the two or more two-dimensional bijective resolution-defined projections is based on an existing two-dimensional rectangular projection having a separable Jacobian.
15. The method of claim 14 , wherein the creating of the two or more two-dimensional bijective resolution-defined projection is further based on a known separable desired resolution distribution.
16. The method of claim 15 , wherein the desired resolution distribution is defined by at least one or more of: a fovea field of view, contrast of the virtual reality image data, movement of the virtual reality image data content, a capturing camera arrangement and geometry, capturing camera lenses, display parameters, and crowd sourced data relating to the virtual reality image content.
17. The method of claim 15 , additionally comprising: compressing at least one of the two or more two-dimensional renditions.
18. The method of claim 15 , wherein at least one of the two or more two-dimensional renditions is compressed at one or more resolutions and/or encoded at one or more resolutions.
19. The method of claim 15 , wherein at least one of the two or more two-dimensional renditions is compressed at one or more bitrates and/or encoded at one or more bitrates.
20. The method of claim 13 , additionally comprising: detecting a user's viewing direction for dynamically selecting at least a portion of the at least one of the two or more two-dimensional renditions according to the detected viewing direction of the user.