Methods and system for generating and displaying 3D videos in a virtual, augmented, or mixed reality environment
Disclosed is an approach for displaying 3D videos in a VR and/or AR system. The 3D videos may include 3D animated objects that escape from the display screen. The 3D videos may interact with objects within the VR and/or AR environment. The 3D video may be interactive with a user such that based on user input corresponding to decisions elected by the user at certain portions of the 3D video such that a different storyline and possibly a different conclusion may result for the 3D video. The 3D video may be a 3D icon displayed within a portal of a final 3D render world.
1. A computer system implementing a virtual reality (VR) and/or augmented reality (AR) video player, comprising:
a computer processor to execute a set of program code instructions; and
a memory to store the program code instructions, in which the program code instructions comprises program code to perform a process for displaying an interactive video in a VR/AR environment comprising:
receiving a video file of an interactive video, the video file comprising:
one or more animation streams,
a data store comprising control data, and
at least one of a video stream and an audio stream,
dynamically generating a control stream based at least in part on the control data and data of a timeline controller, wherein the timeline controller controls a position of the video stream to indicate a current frame of the video that is to be displayed to the user;
requesting a user interaction comprising a user answering a question displayed to the user at a point in time of the video, wherein the control stream controls when to display the question to the user, and the question displayed to the user corresponds to a question that changes a storyline of the interactive video;
receiving an answer to the question via the user interaction;
notifying a timeline controller of the answer;
skipping a portion of the video between a first part of the video and a second part of the video based on the answer; and
displaying the video from the second part of the video.
2. The computer system of claim 1 , wherein the data store is fully loaded into the memory when the video file is received.
3. The computer system of claim 1 , wherein the control stream is fully loaded into the memory when the control stream is generated.
4. The computer system of claim 1 , wherein the interactive video is a stereoscopic 3D video.
5. The computer system of claim 1 , wherein the user interaction comprises at least one of a head pose, an eye tracking, an eye gaze, a hand gesture of the user, a totem gesture, and an object recognizer.
6. The computer system of claim 1 , wherein the program code instructions comprise program code to further perform storing a state of the interactive video with a runtime interactivity data based at least in part on the answer received.
7. The computer system of claim 1 , wherein the interactive video is a stream of non-executable data periodically interrupted by decision nodes associated with different versions of the storyline of the interactive video, and the answer corresponds to a decision elected by the user at a decision node of the interactive video.