Holographically displaying live scenes including three-dimensional objects
Methods, apparatus, devices, subsystems, and systems for holographically displaying live scenes including one or more three-dimensional (3D) objects are provided. In one aspect, a system includes a holographic capturing system and a holographic display system. The holographic capturing system includes: an optical system configured to generate an optical hologram of a live scene that comprises one or more three-dimensional (3D) objects, and an optical sensor configured to capture sequential optical holograms of the live scene and output sequential hologram data associated with the sequential optical holograms of the live scene, each optical hologram being associated with respective hologram data. The holographic display system is configured to optically reconstruct the live scene in a 3D space based on at least part of the sequential hologram data.
1. A system comprising:
a holographic capturing system comprising:
an optical system configured to generate an optical hologram of a live scene that comprises one or more three-dimensional (3D) objects, wherein the optical system comprises a coherent light source and an interferometer, and the optical hologram comprises an interference pattern formed by using the coherent light source and the interferometer; and
an optical sensor configured to capture sequential optical holograms of the live scene and output sequential hologram data associated with the sequential optical holograms of the live scene, each optical hologram being associated with respective hologram data;
a holographic display system configured to optically reconstruct the live scene in a 3D space based on at least part of the sequential hologram data;
a computing device coupled between the holographic capturing system and the holographic display system; and
a frame grabber coupled to the optical sensor and configured to select respective hologram data of one or more optical holograms among the sequential optical holograms to be transmitted to the computing device.
2. The system of claim 1 ,
wherein the computing device is configured to receive the at least part of the sequential hologram data from the optical sensor and generate digital holograms associated with the live scene based on the at least part of the sequential hologram data, and
wherein the holographic display system is configured to receive the digital holograms associated with the live scene from the computing device and diffract light based on the digital holograms to reconstruct the live scene in the 3D space.
3. The system of claim 2 , wherein the holographic capturing system is configured to capture the sequential optical holograms and generate the sequential hologram data, without storing the sequential optical holograms and the sequential hologram data,
wherein the computing device is configured to process the at least part of the sequential hologram data to generate the digital holograms, without storing the at least part of the sequential hologram data and the digital holograms, and
wherein the holographic capturing system, the computing device, and the holographic display system are configured together to capture optical holograms of the live scene and optically reconstruct the live scene in real time.
4. The system of claim 1 , wherein the coherent light source is configured to emit a coherent light beam, and the optical system further comprises:
a beam splitter configured to split the coherent light beam from the coherent light source into an object beam and a reference beam; and
a beam combiner, wherein the live scene is on an optical path of the object beam upstream the beam combiner, and wherein the beam combiner is configured to superimpose the reference beam and the object beam to form the interference pattern.
5. The system of claim 4 , wherein the optical sensor is arranged downstream of the beam combiner and configured to directly capture the interference pattern on an active area of the optical sensor, and
wherein there is no optical lens between the beam combiner and the optical sensor, and wherein there is no optical lens on an optical path of the reference beam between the beam splitter and the beam combiner.
6. The system of claim 4 , wherein the live scene is transmissive, and the object beam travels through the live scene to be incident on the beam combiner.
7. The system of claim 4 , wherein the live scene is reflective, and wherein the optical system comprises one or more reflective mirrors configured to guide the object beam towards the live scene with an angle such that the object beam is reflected or scattered from the live scene to be incident on the beam combiner.
8. The system of claim 4 , wherein the optical system further comprises one or more optical lens on the optical path of the object beam between the live scene and the beam combiner and configured to magnify or demagnify the object beam after interacting with the live scene to be compatible with an active area of the optical sensor.
9. The system of claim 4 , wherein the optical system further comprises an absorber arranged on a side surface of the beam combiner and configured to absorb another part of the reference beam propagating away from the interference pattern.
10. The system of claim 4 , wherein the optical system further comprises one or more optical lens arranged between the beam splitter and the live scene and configured to magnify or demagnify the object beam to be compatible with an active area of the optical sensor.
11. The system of claim 4 , wherein the optical system comprises a collimator arranged upstream the beam splitter and configured to collimate the coherent light beam from the coherent light source.
12. The system of claim 1 , wherein the interference pattern is formed by an object beam from the coherent light source interacting with the live scene and interfering with a reference beam from the coherent light source, and
wherein the optical system further comprises a phase adjuster arranged on an optical path of one of the object beam and the reference beam and configured to dynamically adjust a phase shift of the one of the object beam and the reference beam before the interference pattern is formed, and wherein the phase adjuster comprises a phase shifter or a dynamic retarder.
13. The system of claim 12 , wherein the phase adjuster is configured to sequentially adjust the phase shift to be a series of predetermined values in a time period, such that the optical sensor captures corresponding sequential optical holograms of the live scene in the time period.
14. The system of claim 13 , further comprising a computing device coupled between the holographic capturing system and the holographic display system,
wherein the computing device is configured to generate a digital hologram of the live scene with noise suppression based on the corresponding sequential optical holograms.
15. The system of claim 14 , wherein the phase adjuster comprises a liquid crystal cell configured to adjust the phase shift to be the series of predetermined values by corresponding voltages.
16. The system of claim 15 , wherein the liquid crystal cell is a single cell having a size no smaller than a size of the one of the object beam and the reference beam.
17. The system of claim 15 , wherein a correspondence between the predetermined values of the phase shift and the corresponding voltages is pre-calibrated and predetermined.
18. The system of claim 15 , wherein the computing device is configured to process the corresponding sequential optical holograms to obtain corresponding raw digital holograms, and
wherein the computing device is configured to perform one or more mathematical operations on the corresponding raw digital holograms to generate the digital hologram of the live scene.
19. The system of claim 18 , wherein the series of predetermined values for the phase shift comprises 0, pi/2, pi, 3pi/2, and the digital hologram is calculated based on an expression as follows:
final_hologram=(hologram_0−hologram_pi)/(hologram_pi/2−hologram_3pi/2),
where final_hologram represents the digital hologram,
hologram_0 represents a first corresponding raw digital hologram based on a first corresponding optical hologram with 0 phase shift,
hologram_pi/2 represents a second corresponding raw digital hologram based on a second corresponding optical hologram with pi/2 phase shift,
hologram_pi represents a third corresponding raw digital hologram based on a third corresponding optical hologram with pi phase shift, and
hologram_3pi/2 represents a fourth corresponding raw digital hologram based on a fourth corresponding optical hologram with 3pi/2 phase shift.
20. The system of claim 1 , wherein the optical system comprises a coherent light source that comprises a plurality of coherent light elements, each coherent light element emitting a respective color.
21. The system of claim 20 , wherein the plurality of coherent light elements is configured to sequentially and alternatively emit light with the respective colors, and
wherein the holographic capturing system further comprises a corresponding color filter arranged upstream the optical sensor for each coherent light element of the plurality of coherent light elements, and the corresponding color filter is configured to transmit light with a corresponding color from the coherent light element and block light with other colors to reach the optical sensor.
22. The system of claim 20 , wherein the plurality of coherent light elements is configured to emit light with the respective colors simultaneously, and
wherein the holographic capturing system further comprises a color filter array comprising groups of different color filters on a plurality of sensing pixels of the optical sensor, the different color filters being associated with the respective colors, each group of the different color filters being arranged on a corresponding group of adjacent sensing pixels of the plurality of sensing pixels.
23. The system of claim 22 , wherein the optical sensor is configured to determine hologram data for the respective colors based on a captured optical hologram captured by the corresponding groups of adjacent sensing pixels of the plurality of sensing pixels.
24. The system of claim 22 , further comprising a computing device coupled between the holographic capturing system and the holographic display system,
wherein the computing device is configured to generate digital holograms for the respective colors based on a captured optical hologram captured by the corresponding groups of adjacent sensing pixels of the plurality of sensing pixels.
25. The system of claim 1 , wherein the holographic display system comprises:
a display comprising a plurality of display elements; and
a driving device coupled to the display,
wherein the driving device is configured to:
generate control signals for the plurality of display elements of the display based on a digital hologram associated with the live scene; and
transmit the control signals to the display to modulate the plurality of display elements of the display based on the control signals.
26. The system of claim 25 , further comprising a computing device coupled between the holographic capturing system and the holographic display system,
wherein the computing device is configured to receive the at least part of the sequential hologram data from the optical sensor and generate digital holograms associated with the live scene based on the at least part of the sequential hologram data, and
wherein the holographic display system is configured to receive the digital holograms associated with the live scene from the computing device and diffract light based on the digital holograms to reconstruct the live scene in the 3D space.
27. The system of claim 26 , wherein the digital holograms comprise a series of groups of digital holograms for a plurality of colors, and wherein the holographic display system further comprises an illuminator comprising a plurality of coherent light elements for the plurality of colors, and
wherein the driving device is configured to:
sequentially modulate the display with a first digital hologram for a first color during a first time period and modulate the display with a second digital hologram for a second color during a second, sequential time period; and
control the illuminator to sequentially turn on a first coherent light element to emit light with the first color during the first time period and a second coherent light element to emit light with the second color during the second, sequential time period.
28. The system of claim 26 , wherein the driving device comprises at least one of
a display driver coupled to the display,
an illuminator driver coupled to the illuminator, or
a memory coupled to at least one of the display driver or the illuminator driver.
29. The system of claim 1 , wherein the reconstructed live scene in the 3D space has one or more holography characteristics comprising occlusion, parallax, and accommodation.
30. A system comprising:
a holographic capturing system comprising:
an optical system configured to generate an optical hologram of a live scene that comprises one or more three-dimensional (3D) objects, wherein the optical system comprises a coherent light source and an interferometer, and the optical hologram comprises an interference pattern formed by using the coherent light source and the interferometer; and
an optical sensor configured to capture sequential optical holograms of the live scene and output sequential hologram data associated with the sequential optical holograms of the live scene, each optical hologram being associated with respective hologram data;
a holographic display system configured to optically reconstruct the live scene in a 3D space based on at least part of the sequential hologram data; and
a computing device coupled between the holographic capturing system and the holographic display system,
wherein the holographic capturing system is configured to capture the sequential optical holograms and generate the sequential hologram data, without storing the sequential optical holograms and the sequential hologram data,
wherein the computing device is configured to receive the at least part of the sequential hologram data from the optical sensor and generate digital holograms associated with the live scene based on the at least part of the sequential hologram data, without storing the at least part of the sequential hologram data and the digital holograms,
wherein the holographic display system is configured to receive the digital holograms associated with the live scene from the computing device and diffract light based on the digital holograms to reconstruct the live scene in the 3D space, and
wherein the holographic capturing system, the computing device, and the holographic display system are configured together to capture optical holograms of the live scene and optically reconstruct the live scene in real time.
31. A system comprising:
a holographic capturing system comprising:
an optical system configured to generate an optical hologram of a live scene that comprises one or more three-dimensional (3D) objects, wherein the optical system comprises a coherent light source and an interferometer, and the optical hologram comprises an interference pattern formed by using the coherent light source and the interferometer; and
an optical sensor configured to capture sequential optical holograms of the live scene and output sequential hologram data associated with the sequential optical holograms of the live scene, each optical hologram being associated with respective hologram data;
a holographic display system configured to optically reconstruct the live scene in a 3D space based on at least part of the sequential hologram data; and
a computing device coupled between the holographic capturing system and the holographic display system,
wherein the computing device is configured to receive the at least part of the sequential hologram data from the optical sensor and generate digital holograms associated with the live scene based on the at least part of the sequential hologram data,
wherein the holographic display system is configured to receive the digital holograms associated with the live scene from the computing device and diffract light based on the digital holograms to reconstruct the live scene in the 3D space,
wherein the holographic display system comprises: a display comprising a plurality of display elements; and a driving device coupled to the display, wherein the driving device is configured to: generate control signals for the plurality of display elements of the display based on a digital hologram associated with the live scene; and transmit the control signals to the display to modulate the plurality of display elements of the display based on the control signals,
wherein the digital holograms comprise a series of groups of digital holograms for a plurality of colors, and wherein the holographic display system further comprises an illuminator comprising a plurality of coherent light elements for the plurality of colors, and
wherein the driving device is configured to:
sequentially modulate the display with a first digital hologram for a first color during a first time period and modulate the display with a second digital hologram for a second color during a second, sequential time period; and
control the illuminator to sequentially turn on a first coherent light element to emit light with the first color during the first time period and a second coherent light element to emit light with the second color during the second, sequential time period.