Holographic video capture and telepresence system
The invention is directed to recording, transmitting, and displaying a three-dimensional image of a face of a user in a video stream. Reflected light from a curved or geometrically shaped screen is employed to provide multiple perspective views of the user's face that are transformed into the image, which is communicated to remotely located other users. A head mounted projection display system is employed to capture the reflective light. The system includes a frame, that when worn by a user, wraps around and grips the user's head. Also, at least two separate image capture modules are included on the frame and generally positioned relatively adjacent to the left and right eyes of a user when the system is worn. Each module includes one or more sensor components, such as cameras, that are arranged to detect at least reflected non-visible light from a screen positioned in front of the user.
1. A system for displaying an image, comprising:
an apparatus that includes two or more light sensor devices positioned on opposite sides of a user;
a display surface; and
one or more processors that execute instructions to perform actions, including:
employing one or more light sensor devices to detect reflected light that is used to generate two or more views of the user, wherein one or more portions of the reflected light are reflected from one or more of the display surface or a surface of the user;
generating the image based on the two or more views of the user; and
in response to the user leaning their head towards a focal zone of the display surface and passing through a visual feedback cue, the image is displayed in a virtual environment to one or more other users until the user leans their head away from the focal zone of the display surface and passes outside the visual feedback cue.
2. The system of claim 1 , wherein the one or more light sensor devices are arranged to detect one or more of non-visible reflected light or visible reflected light.
3. The system of claim 1 , wherein the display surface further comprises a shape that is one or more of non-flat, concave, folded, parabolic, triptych, cylindrical, curved, elliptical, toroid, or elliptical parabolic.
4. The system of claim 1 , wherein the two or more views of the user include one or more perspective views that are employed to generate a video of the user for display to the one or more other users.
5. The system of claim 1 , further comprising one or more light scanners that are arranged to direct light onto the display surface, wherein one or more portions of the directed light is reflected from the display surface onto the surface of the user.
6. The system of claim 1 , wherein the two or more views of the user are employed to generate a three-dimensional (3-D) avatar of the user in real time that is displayed to the one or more other users.
7. The system of claim 1 , further comprising:
one or more light scanners that are arranged to direct near infrared (NIR) light onto the surface of the user, wherein one or more portions of the NIR light are reflected from the surface of the user, and wherein the one or more portions of the reflected NIR light is employed to color correct the two or more views of the user, including one or more of a skin color, blood oxygenation level, blushing, or a tan.
8. The system of claim 1 , wherein the display surface further comprises one or more of:
a transparent screen positioned for total internal reflection;
a translucent screen arranged with a retro-reflective coating; or
one or more windows of a vehicle.
9. An apparatus for capturing an image of a user for display, comprising:
two or more light sensor devices positioned on opposite sides of the user; and
one or more processors that execute instructions to perform actions, including:
employing one or more light sensor devices to detect reflected light that is used to generate two or more views of the user, wherein one or more portions of the reflected light are reflected from one or more of the display surface or a surface of the user;
generating the image based on the two or more views of the user; and
in response to the user leaning their head towards a focal zone of the display surface and passing through a visual feedback cue, the image is displayed in a virtual environment to one or more other users until the user leans their head away from the focal zone of the display surface and passes outside the visual feedback cue.
10. The apparatus of claim 9 , wherein the one or more light sensor devices are arranged to detect one or more of non-visible reflected light or visible reflected light.
11. The apparatus of claim 9 , wherein the display surface further comprises a shape that is one or more of non-flat, concave, folded, parabolic, triptych, cylindrical, curved, elliptical, toroid, or elliptical parabolic.
12. The apparatus of claim 9 , wherein the two or more views of the user include one or more perspective views that are employed to generate a video of the user for display to the one or more other users.
13. The apparatus of claim 9 , further comprising one or more light scanners that are arranged to direct light onto the display surface, wherein one or more portions of the directed light is reflected from the display surface onto the surface of the user.
14. The apparatus of claim 9 , wherein the two or more views of the user are employed to generate a three-dimensional (3-D) avatar of the user in real time that is displayed to the one or more other users.
15. The apparatus of claim 9 , further comprising:
one or more light scanners that are arranged to direct near infrared (NIR) light onto the surface of the user, wherein one or more portions of the NIR light are reflected from the surface of the user, and wherein the one or more portions of the reflected NIR light is employed to color correct the two or more views of the user, including one or more of a skin color, blood oxygenation level, blushing, or a tan.
16. The apparatus of claim 9 , wherein the display surface further comprises one or more of:
a transparent screen positioned for total internal reflection;
a translucent screen arranged with a retro-reflective coating; or
one or more windows of a vehicle.
17. A method for displaying an image of a user by employing one or more processors that execute instructions to perform actions, comprising:
employing two or more light sensor devices to detect reflected light that is used to generate two or more views of the user, wherein the two or more light sensor devices are positioned on opposite sides of a head of the user, and wherein one or more portions of the reflected light are reflected from one or more of a display surface or a surface of the user;
generating the image based on the two or more views of the user; and
in response to the user leaning their head towards a focal zone of the display surface and passing through a visual feedback cue, the image is displayed in a virtual environment to one or more other users until the user leans their head away from the focal zone of the display surface and passes outside the visual feedback cue.
18. The method of claim 17 , wherein the two or more views of the user include one or more perspective views that are employed to generate a video of the user for display to the one or more other users.
19. A computer readable non-transitory storage media that includes instructions wherein execution of the instructions by one or more processors performs actions for displaying an image of a user, comprising:
employing two or more light sensor devices to detect reflected light that is used to generate two or more views of the user, wherein the two or more light sensor devices are positioned on opposite sides of a head of the user, and wherein one or more portions of the reflected light are reflected from one or more of a display surface or a surface of the user;
generating the image based on the two or more views of the user; and
in response to the user leaning their head towards a focal zone of the display surface and passing through a visual feedback cue, the image is displayed in a virtual environment to one or more other users until the user leans their head away from the focal zone of the display surface and passes outside the visual feedback cue.
20. The media of claim 19 , wherein the two or more views of the user include one or more perspective views that are employed to generate a video of the user for display to the one or more other users.