Methods and system for generating and displaying 3D videos in a virtual, augmented, or mixed reality environment
Disclosed is an approach for displaying 3D videos in a VR and/or AR system. The 3D videos may include 3D animated objects that escape from the display screen. The 3D videos may interact with objects within the VR and/or AR environment. The 3D video may be interactive with a user such that based on user input corresponding to decisions elected by the user at certain portions of the 3D video such that a different storyline and possibly a different conclusion may result for the 3D video. The 3D video may be a 3D icon displayed within a portal of a final 3D render world.
1. A method comprising:
receiving a video file of a video, the video file comprising:
one or more animation streams;
a data store comprising control data; and
at least one of a video stream or an audio stream;
dynamically generating a control stream from the control data and a timeline controller;
loading a model of a 3D object received from the data store; and
attaching the 3D object to an animation stream of the one or more animation streams,
wherein the model of the 3D object is loaded based at least in part on the control stream looking ahead in time of the video streams and anticipating when the 3D object needs to be displayed.
2. The method of claim 1 , wherein the one or more animation streams correspond to respective 3D models stored within a data store.
3. The method of claim 1 , wherein the one or more animation streams control movements, orientation and positions of 3D objects relative to the video.
4. The method of claim 1 , further comprising determining a lead time for loading the model is based at least on one of a size of the model, a network bandwidth, or processing power of a user's virtual and/or augmented reality system.
5. The method of claim 1 , further comprising displaying the video via a virtual and/or augmented reality device.
6. The method of claim 1 , wherein the data store is fully loaded into memory when the video file is received.
7. The method of claim 1 , wherein the control stream is fully loaded into memory when the control stream is generated.
8. The method of claim 1 , wherein the video is a stereoscopic 3D video.
9. A computer system implementing a mixed reality video player, comprising:
a computer processor to execute a set of program code instructions; and
a memory to hold the program code instructions, in which the program code instructions comprises program code to perform:
receiving a video file of a video, the video file comprising:
one or more animation streams,
a data store comprising control data, and
at least one of a video stream or an audio stream,
dynamically generating a control stream from the control data and a timeline controller,
loading a model of a 3D object received from the data store, and
attaching the 3D object to an animation stream of the one or more animation streams,
wherein the model of the 3D object is loaded based at least in part on the control stream looking ahead in time of the video streams and anticipating when the 3D object needs to be displayed.
10. The computer system of claim 9 , wherein the one or more animation streams correspond to respective 3D models stored within a data store.
11. The computer system of claim 9 , wherein the one or more animation streams control movements, orientation and positions of 3D objects relative to the video.
12. The computer system of claim 9 , wherein the program code instructions comprise program code to further perform determining a lead time for loading the model is based at least on one of a size of the model, a network bandwidth, or processing power of a user's virtual and/or augmented reality system.
13. The computer system of claim 9 , wherein the program code instructions comprise program code to further perform displaying the video via a virtual and/or augmented reality device.
14. The computer system of claim 9 , wherein the data store is fully loaded into memory when the video file is received.
15. The computer system of claim 9 , wherein the control stream is fully loaded into memory when the control stream is generated.
16. The computer system of claim 9 , wherein the video is a stereoscopic 3D video.
17. A method comprising:
receiving a video file of a video, the video file comprising
one or more animation streams;
a data store comprising control data; and
at least one of a video stream or an audio stream;
dynamically generating a control stream from the control data and a timeline controller;
requesting a user interaction answering a question displayed to the user at a point in time of the video;
receiving an answer to the question via the user interaction;
notifying a timeline controller of the answer;
skipping to an appropriate portion of the video corresponding to the answer; and
displaying the video from the appropriate portion.