Providing a collaborative immersive environment using a spherical camera and motion capture

1. A system for providing a simulated environment and telepresence using real-world video, the system comprising:

a spherical camera positioned to capture real-world video at a first location;

one or more head-mounted display devices at a second location, the second location remote from the first location, the one or more head-mounted display devices being positioned to display the captured real-world video from the first location and also to display a simulation corresponding to the real-world video;

one or more computers to support the one or more head-mounted displays;

a motion capture system being positioned to capture head rotation information for one or more users to control a pan, tilt, and zoom of the real-world video responsive to the captured head rotation information for the one or more users so that when a position of a user's head changes, the portion of the real-world video displayed in the head-mounted display changes accordingly;

a virtual reality simulator being positioned to receive CAD design data from a CAD design and to create a three-dimensional virtual reality simulation from the CAD design data so that the one or more head-mounted display devices display the simulation: and

a computer program product, stored on a computer readable memory media, operated on a computer, the computer program product comprising a set of instructions that, when executed by the computer, cause the computer to perform the operations of:

receiving user input to select between displaying the real-world video and the simulation in the head-mounted display.

2. A system of claim 1 , wherein the spherical camera positioned to capture real-world video at the first location includes six digital cameras embedded into a single device so that the single device captures about 75% of a sphere.

3. A system of claim 1 , wherein the operations further comprise reading the video files from the spherical camera into buffers and indexing the files so that a video group with many files is played as a single video file.

4. A system of claim 1 , wherein the operations further comprise superimposing a portion of the simulation representing a field of view onto the portion of the real-world video rendered in the head-mounted display.

5. A system of claim 1 , wherein the motion capture system includes up to 24 cameras mounted on a truss and, for each of the one or more users, gloves and a body suit with markers at predetermined locations on the suit.

6. The system of claim 1 , wherein the CAD design data is for an aircraft.

7. A system for providing a simulated environment and telepresence using real-world video, the system comprising:

a spherical camera positioned to capture real-world video at a first location;

an immersive observation system at a second location, including one or more computers and a reconfigurable room with displays on three walls and the floor, the second location remote from the first location, the displays being positioned to display the captured real-world video from the first location and also to display a simulation corresponding to the real-world video;

a motion capture system being positioned to capture head rotation information for one or more users to control a pan, tilt, and zoom of the real-world video responsive to the captured head rotation information for the one or more users so that when a position of a user's head changes, the portion of the real-world video displayed in the head-mounted display changes accordingly;

a virtual reality simulator being positioned to receive CAD design data from a CAD design and to create a three-dimensional virtual reality simulation from the CAD design data so that the one or more head-mounted display devices display the simulation: and

a computer program product, stored on a computer readable memory media, operated on a computer, the computer program product comprising a set of instructions that, when executed by the computer, cause the computer to perform the operations of:

receiving user input to select between displaying the real-world video and the simulation in the head-mounted display.

8. A system of claim 7 , wherein the spherical camera positioned to capture real-world video at the first location includes six digital cameras embedded into a single device so that the single device captures about 75% of a sphere.

9. A system of claim 7 , wherein the operations further comprise reading the video tiles from the spherical camera into buffers and indexing the files so that a video group with many files is played as a single video file.

10. A system of claim 7 , wherein the operations further comprise superimposing a portion of the simulation representing a field of view onto the portion of the real-world video.

11. The system of claim 7 , wherein the CAD design data is for an aircraft.

12. A method of displaying a simulation with real-world video using immersive technology, the method comprising:

capturing real-world video by a spherical camera at a first location:

rendering to a user at a second location a portion of the real-world video in a head-mounted display, the second location remote from the first location, the portion of the real-world video representing a field of view, the head-mounted display being positioned to display the real-world video and to display a simulation related to the real-world video;

capturing head rotation information of the user by a motion capture system;

controlling a pan, tilt, and zoom of the real-world video responsive to the captured head rotation information of the user so that when a position of the user's head changes, the portion of the real-world video rendered in the head-mounted display changes accordingly;

receiving, by a virtual reality simulator, CAD design data from a CAD design to create a three-dimensional virtual reality simulation from the CAD design data so that the head-mounted display displays the simulation: and

switching, under user control, between displaying the real-world video and the simulation in the head-mounted display.

13. A method of claim 12 , wherein the step of rendering to a user at a second location a portion of the real-world video in a head-mounted display involves reading video files from the spherical camera into buffers and indexing the files so that a video group with many files is played as a single video file.

14. A method of claim 12 , further comprising superimposing a portion of the simulation representing a field of view onto the portion of the real-world video rendered in the head-mounted display.

15. A method of claim 12 , further comprising comparing the simulation with the real-world video to validate the simulation to thereby provide feedback on the simulation.

16. A method of claim 12 , further comprising, training the user at the second location to perform one or more tasks at an environment substantially similar to the environment of the first location.

17. A method of claim 12 , wherein the user is one user of a plurality of users at the second location, and wherein each user of the plurality of users is simultaneously rendered a user-controlled portion of the real-world video in a head-mounted display associated with said user so that each user has a unique field of view.

18. The method of claim 17 , wherein the motion capture system supports at least 6 users, and wherein the head-mounted display provides to the user a stereoscopic field of view.