Efficient capture and delivery of walkable and interactive virtual reality or 360 degree video
Disclosed are systems and methods for generating a walkable 360-degree video or virtual reality (VR) environment. 360-degree video data is obtained for a real-world environment and comprises a plurality of chronologically ordered frames captured by traversing a first path through the real-world environment. One or more processing operations are applied to generate a processed 360-degree video, which can be displayed to a user of an omnidirectional treadmill. Locomotion information is received from one or more sensors of the omnidirectional treadmill, wherein the locomotion information is generated based on a physical movement on or within the omnidirectional treadmill. Using the received locomotion information, one or more playback commands for controlling playback of the processed 360-degree video are generated. One or more selected frames of the processed 360-degree video are rendered for presentation and display to the user, based on the one or more playback commands.
1. A method comprising:
obtaining a 360-degree video data of a real-world environment, wherein the 360-degree video data comprises a plurality of chronologically ordered frames captured by traversing a plurality of paths through the real-world environment;
processing the 360-degree video data to generate a plurality of processed 360-degree videos, wherein the plurality of processed 360-degree video data includes a plurality of virtual paths corresponding with the plurality of paths through the real-world environment and one or more junction points corresponding to locations where virtual paths of the plurality of virtual paths intersect;
receiving locomotion information from one or more sensors, wherein the locomotion information is generated based on physical movement of a user within an omnidirectional treadmill;
generating, using the received locomotion information, one or more playback commands for controlling playback of the plurality of processed 360-degree videos; and
rendering one or more frames of the plurality of processed 360-degree videos based on the one or more playback commands.
2. The method of claim 1 , further comprising:
generating, based on the plurality of processed 360-degree videos, a walkable virtual reality (VR) environment including the plurality of virtual paths.
3. The method of claim 1 , wherein the received locomotion information includes a direction of locomotion on the omnidirectional treadmill.
4. The method of claim 1 , wherein the received locomotion information includes a speed of locomotion.
5. The method of claim 1 , wherein the one or more playback commands includes adjustments to a playback speed of the plurality of processed 360-degree videos.
6. The method of claim 1 , wherein an elapsed time between rendering the one or more frames is proportional to the speed of locomotion.
7. The method of claim 1 , further comprising:
calculating a direction of virtual movement along a virtual path within the walkable VR environment, the calculating based on the received locomotion information; and
obtaining the one or more frames from the plurality of processed 360-degree videos in response to a determination that the calculated direction of the virtual movement corresponds to a traversal direction in which the 360-degree video data was captured along a path of the plurality of paths through the real-world environment.
8. The method of claim 1 , wherein processing the 360-degree video data to generate the processed 360-degree video data comprises performing one or more of:
stabilization;
top and tailing;
cropping;
adjusting playback speed; and
mirroring.
9. A system comprising:
at least one processor; and
a memory having instructions therein, which when executed by the at least one processor cause the system to:
obtain 360-degree video data of a real-world environment, wherein the 360-degree video data comprises a plurality of chronologically ordered frames captured by traversing a plurality of paths through the real-world environment;
process the 360-degree video data to generate a plurality of processed 360-degrees videos, wherein the plurality of processed 360-degree video data includes a plurality of virtual paths corresponding with the plurality of paths through the real-world environment and one or more junction points corresponding to locations where virtual paths of the plurality of virtual paths intersect;
receive locomotion information from one or more sensors, wherein the locomotion information is generated based on physical movement of a user within an omnidirectional treadmill;
generate, using the received locomotion information, one or more playback commands for controlling playback of the plurality of processed 360-degree videos; and
render one or more frames of the plurality of processed 360-degree videos based on the one or more playback commands.
10. The system of claim 9 further comprising instructions, which when executed by the at least one processor, causes the system to:
generate, based on the plurality of processed 360-degree videos, a walkable virtual reality (VR) environment including the plurality of virtual paths.
11. The system of claim 9 , wherein the received locomotion information includes a direction of locomotion on the omnidirectional treadmill.
12. The system of claim 9 , wherein the received locomotion information includes a speed of locomotion.
13. The system of claim 9 , wherein the one or more playback commands includes adjustments to a playback speed of the plurality of processed 360-degree videos.
14. The system of claim 9 , wherein an elapsed time between rendering the one or more frames is proportional to the speed of locomotion.
15. The system of claim 9 , further comprising instructions, which when executed by the at least one processor, causes the system to:
calculate a direction of virtual movement along the first virtual path within the walkable VR environment, the calculating based on the received locomotion information; and
obtain the one or more frames from the plurality of processed 360-degree videos in response to a determination that the calculated direction of the virtual movement corresponds to a traversal direction in which the 360-degree video data was captured along a path of the plurality of paths through the real-world environment.
16. The system of claim 9 , wherein processing the 360-degree video data to generate the processed 360-degree video data comprises performing one or more of:
stabilization;
top and tailing;
cropping;
adjusting playback speed; and
mirroring.
17. At least one non-transitory computer-readable medium storing instructions therein, which when executed by at least one processor cause the at least one processor to:
obtain 360-degree video data of a real-world environment, wherein the 360-degree video data comprises a plurality of chronologically ordered frames captured by traversing a plurality of paths through the real-world environment;
process the 360-degree video data to generate a plurality of processed 360-degrees videos, wherein the plurality of processed 360-degree video data includes a plurality of virtual paths corresponding with the plurality of paths through the real-world environment and one or more junction points corresponding to locations where virtual paths of the plurality of virtual paths intersect;
receive locomotion information from one or more sensors, wherein the locomotion information is generated based on physical movement of a user within an omnidirectional treadmill;
generate, using the received locomotion information, one or more playback commands for controlling playback of the plurality of processed 360-degree videos; and
render one or more frames of the plurality of processed 360-degree videos based on the one or more playback commands.
18. The at least one non-transitory computer-readable medium of claim 17 , further comprising instructions, which when executed by the at least one processor, causes the at least one processor to:
generate, based on the plurality of processed 360-degree videos, a walkable virtual reality (VR) environment including the plurality of virtual paths.
19. The at least one non-transitory computer-readable medium of claim 17 , wherein the received locomotion information includes a direction of locomotion on the omnidirectional treadmill.
20. The at least one non-transitory computer-readable medium of claim 17 , wherein the received locomotion information includes a speed of locomotion.