Method And Apparatus For Real-time Data Communication in Full-Presence Immersive Platforms
Real-time two-way communication between a user immersed in a virtual reality (VR) environment and another party who is not in the VR environment. The sequence of video frames being presented to the VR user on, e.g., a head mounted display, i.e., the VR user's “field of view” in the VR environment, is shown to a guest user on their computing device. Simultaneously, video from a camera in a guest user computing device is shown to the VR user in a window in the VR environment. Audio is also exchanged in real time between the VR user and guest user by use of a microphone and loudspeaker in both the VR system and the guest user computing device. The VR user's system can also support both rotational and translational user movement thus properly changing the VR user's apparent distance from objects displayed in the VR environment.
1 . A method for a user wearing a head-mounted display (HMD) to experience movement with six degrees of freedom in a virtual reality environment, the HMD covering the user's field of vision and having a HMD tracker which provides information regarding the position and orientation of the HMD, comprising:
determining, by a processor, an initial position and orientation of the user using information from the HMD tracker;
generating, by the processor, a three-dimensional virtual reality environment configured as a sphere of a predefined radius around the user with three-dimensional virtual objects located on the surface of the sphere, each virtual object having an initial apparent depth from the determined initial position and orientation of the user;
generating and outputting, by the processor, instructions to the HMD to display a portion of the virtual reality environment that is within the user's field of view based upon the determined initial position and orientation of the HMD;
determining, by the processor, using information from the HMD tracker that the user has moved rotationally;
generating and outputting, by the processor, instructions to the HMD to display a different portion of the virtual reality environment that is within the user's field of view after the user's rotational movement relative to the sphere;
determining, by the processor, using information from the HMD tracker that the user has moved translationally; and
generating and outputting, by the processor, instructions to the HMD to display a different portion of the virtual reality environment that is within the user's field of view after the user's translational movement relative to the sphere and to display any virtual objects within the user's field of view at a new apparent position and depth corresponding to the user's translational movement relative to the sphere.
2 . The method of claim 1 wherein the predefined diameter of the sphere is between approximately 2 and 20 meters.
3 . The method of claim 5 wherein the predefined diameter of the sphere is approximately 6 to 7 meters.
4 . The method of claim 1 wherein one or more of the three-dimensional virtual objects are representations of physical objects, menus, dialog boxes, a digital floor or platform, windows containing video streams, or avatars of other virtual reality users.
5 . A system for a user to experience movement with six degrees of freedom in a virtual reality environment, comprising:
a head-mounted display (HMD) covering the user's field of vision and having a HMD tracker which provides information regarding the position and orientation of the HMD;
a processor configured to:
determine an initial position and orientation of the user using information from the HMD tracker;
generate a three-dimensional virtual reality environment configured as a sphere of a predefined radius around the user with three-dimensional virtual objects located on the surface of the sphere, each virtual object having an initial apparent depth from the determined initial position and orientation of the user;
generate and output instructions to the HMD to display a portion of the virtual reality environment that is within the user's field of view based upon the determined initial position and orientation of the user's head;
determine using information from the HMD tracker that the user has moved rotationally;
generate and output instructions to the HMD to display a different portion of the virtual reality environment that is within the user's field of view after the user's rotational movement relative to the sphere;
determine using information from HMD the tracker that the user has moved translationally;
generate and output instructions to the HMD to display a different portion of the virtual reality environment that is within the user's field of view after the user's translational movement relative to the sphere and to display any virtual objects within the user's field of view at a new apparent position and depth corresponding to the user's translational movement relative to the sphere.
6 . The system of claim 1 wherein the predefined diameter of the sphere is between approximately 2 and 20 meters.
7 . The system of claim 5 wherein predefined diameter of the sphere is approximately 6 to 7 meters.
8 . The system of claim 1 wherein one or more of the three-dimensional virtual objects are representations of physical objects, menus, dialog boxes, a digital floor or platform, windows containing video streams, or avatars of other virtual reality users.
9 . A non-transitory computer readable storage medium having embodied thereon instructions for causing a computing device to execute a method for a user wearing a head-mounted display (HMD) to experience movement with six degrees of freedom in a virtual reality environment, the HMD covering the user's field of vision and having a HMD tracker which provides information regarding the position and orientation of the HMD, the method comprising:
determining, by a processor, an initial position and orientation of the user using information from the HMD tracker;
generating, by the processor, a three-dimensional virtual reality environment configured as a sphere of a predefined radius around the user with three-dimensional virtual objects located on the surface of the sphere, each virtual object having an initial apparent depth from the determined initial position and orientation of the user;
generating and outputting, by the processor, instructions to the HMD to display a portion of the virtual reality environment that is within the user's field of view based upon the determined initial position and orientation of the HMD;
determining, by the processor, using information from the HMD tracker that the user has moved rotationally;
generating and outputting, by the processor, instructions to the HMD to display a different portion of the virtual reality environment that is within the user's field of view after the user's rotational movement relative to the sphere; and
determining, by the processor, using information from the HMD tracker that the user has moved translationally;
generating and outputting, by the processor, instructions to the HMD to display a different portion of the virtual reality environment that is within the user's field of view after the user's translational movement relative to the sphere and to display any virtual objects within the user's field of view at a new apparent position and depth corresponding to the user's translational movement relative to the sphere.
10 . The non-transitory computer readable storage medium of claim 9 , wherein the predefined diameter of the sphere is between approximately 2 and 20 meters.
11 . The non-transitory computer readable storage medium of claim 9 , wherein the predefined diameter of the sphere is approximately 6 to 7 meters.
12 . The non-transitory computer readable storage medium of claim 9 , wherein one or more of the three-dimensional virtual objects are representations of physical objects, menus, dialog boxes, a digital floor or platform, windows containing video streams, or avatars of other virtual reality users.