Method and system for predictive processing of virtual reality content
Aspects of the present disclosure include a method and system for the predictive processing of virtual reality content that enables a more immersive experience to the user by adaptively changing parameters of virtual reality content using desired resolution distributions. in some embodiments, the desired resolution distributions can be used in generating resolution-defined projections. The resolution-defined projections can be two-dimensional and virtual reality content agnostic. According to some aspects, the projections can be used to generate virtual reality content images and video that can be encoded and/or compressed using predictive processing and transmission with lower or without increasing bandwidth requirements, and to be perceived of a higher resolution. Additional aspects include, the adaptive focus via the resolution-defined projections using the user's viewing direction and/or orientation, optimization data from a feedback loop and/or crowdsourcing data.
1. A system for predictive processing and delivering virtual reality content, the system comprising:
one or more computer servers comprising one or more processors and one or more storage devices; and
executable software stored on at least one of the one or more storage devices and executable on demand, the software operative with at least one of the one or more processors to cause the one or more computer servers to:
process feedback data relating to one or more events related to a virtual reality content;
receive sensor data to determine one or more of a viewing direction or an orientation of a user;
additionally receive data descriptive of the virtual reality content;
predict, based at least in part on the feedback data, a change in one or more of the determined viewing direction or the determined orientation;
generate one or more two-dimensional renditions from the virtual reality content, wherein the one or more two-dimensional renditions are generated by mapping the virtual reality content to a two-dimensional bijective resolution-defined projection, wherein each of the one or more two-dimensional renditions represents a full sphere of the virtual reality content;
select at least one of the one or more two-dimensional renditions for display; and
cause display, by a display device, an inverse projection of the virtual reality content to a sphere or other three-dimensional shape,
wherein one or more of the generating the one or more two-dimensional renditions or the selecting at least one of the one or more two-dimensional renditions for display is based at least in part on the predicted change in one or more of the determined viewing direction or the determined orientation.
2. The system of claim 1 , wherein the feedback data comprises one or more of pre-recorded data on the one or more storage devices based on feedback received from the user during at least one previous display of virtual reality content or crowdsourced data relating to a location of the virtual reality content.
3. The system of claim 1 , wherein the feedback data comprises pre-recorded data on the one or more storage devices based on feedback received from the user during at least one previous display of virtual reality content.
4. The system of claim 1 , wherein the feedback data comprises crowdsourced data recorded on the one or more storage devices relating to a location of the virtual reality content.
5. A computer-implemented method for predictive processing and delivering virtual reality content, the method comprising:
processing, by a computer system, feedback data relating to one or more events related to a virtual reality content;
receiving, by the computer system, sensor data to determine a viewing direction of a user;
additionally receiving, by the computer system, data descriptive of a location of the virtual reality content;
predicting, by the computer system, based at least in part on the feedback data, a change in the determined viewing direction of the user;
generating, by the computer system, one or more two-dimensional renditions from the virtual reality content, wherein the one or more two-dimensional renditions are generated by mapping the virtual reality content to a two-dimensional bijective resolution-defined projection, wherein each of the one or more two-dimensional renditions represents a full sphere of the virtual reality content;
selecting, by the computer system, at least one of the one or more two-dimensional renditions for display; and
causing, by the computer system, display on a display device an inverse projection of the virtual reality content to a sphere or other three-dimensional shape,
wherein one or more of the generating the one or more two-dimensional renditions or the selecting at least one of the one or more two-dimensional renditions for display is based at least in part on the predicted change in the determined viewing direction, and
wherein the computer system comprises a computer processor and an electronic storage medium.
6. The computer-implemented method of claim 5 , wherein the feedback data comprises one or more of pre-recorded data based on feedback received from the user during at least one previous display of virtual reality content or crowdsourced data relating to the location of the virtual reality content.
7. The computer-implemented method of claim 5 , wherein the two dimensional bijective resolution-defined projection is created according to a desired resolution distribution, wherein the desired resolution distribution is defined by at least one or more of: a fovea field of view, contrast of the virtual reality content, movement of the virtual reality content, arrangement and geometry of a capturing camera, lens of a capturing camera, display parameters, or crowd sourced data relating to the virtual reality content.
8. The computer-implemented method of claim 7 further comprising one or more of:
compressing or encoding at least one or more of the two-dimensional renditions based at least in part on the predicted change in the determined viewing direction at one or more resolutions.
9. The computer-implemented method of claim 7 , further comprising one or more of:
compressing or encoding at least one or more of the two-dimensional renditions based at least in part on the predicted change in the determined viewing direction at one or more bitrates.
10. The computer-implemented method of claim 5 , wherein the feedback data comprises pre-recorded data based on feedback received from the user during at least one previous display of virtual reality content.
11. The computer-implemented method of claim 5 , wherein the feedback data comprises crowdsourced data relating to the location of the virtual reality content.
12. A computer-implemented method for predictive processing and delivering virtual reality content, the method comprising:
processing, by a computer system, feedback data relating to one or more events related to a virtual reality content;
additionally receiving, by the computer system, data descriptive of the virtual reality content;
identifying, by the computer system, based at least in part on the feedback data and the data descriptive of the virtual reality content, at least one area of interest;
generating, by the computer system, one or more two-dimensional renditions from the virtual reality content, wherein the one or more two-dimensional renditions are generated by mapping the virtual reality content to a two-dimensional bijective resolution-defined projection, wherein each of the one or more two-dimensional renditions represents a full sphere of the virtual reality content;
selecting, by the computer system, at least one of the one or more two-dimensional renditions for display; and
causing, by the computer system, display on a display device an inverse projection of the virtual reality content to a sphere or other three-dimensional shape,
wherein one or more of the generating the one or more two-dimensional renditions or the selecting at least one of the one or more two-dimensional renditions for display is based at least in part on the identified area of interest, and
wherein the computer system comprises a computer processor and an electronic storage medium.
13. The computer-implemented method of claim 12 , wherein the one or more two-dimensional bijective resolution-defined projection is created according to a desired resolution distribution.
14. The computer-implemented method of claim 13 , wherein the desired resolution distribution is defined by at least one or more of: a fovea field of view, contrast of the virtual reality content, movement of the virtual reality content, arrangement and geometry of a capturing camera, lens of a capturing camera, display parameters, or crowd sourced data relating to the virtual reality content.
15. The computer-implemented method of claim 12 , wherein the feedback data comprises one or more of pre-recorded data based on feedback received from the user during at least one previous display of virtual reality content or crowdsourced data relating to a location of the virtual reality content.
16. The computer-implemented method of claim 12 , wherein the feedback data comprises pre-recorded data based on feedback received from the user during at least one previous display of virtual reality content.
17. The computer-implemented method of claim 12 , wherein the feedback data comprises crowdsourced data relating to a location of the virtual reality content.
18. The computer-implemented method of claim 12 , wherein the area of interest is determined by pre-recorded determined viewing directions relating to a location of the virtual reality content.