Holographically displaying live scenes including three-dimensional objects
Methods, apparatus, devices, subsystems, and systems for holographically displaying live scenes including one or more three-dimensional (3D) objects are provided. In one aspect, a system includes a holographic capturing system and a holographic display system. The holographic capturing system includes: an optical system configured to generate an optical hologram of a live scene that comprises one or more three-dimensional (3D) objects, and an optical sensor configured to capture sequential optical holograms of the live scene and output sequential hologram data associated with the sequential optical holograms of the live scene, each optical hologram being associated with respective hologram data. The holographic display system is configured to optically reconstruct the live scene in a 3D space based on at least part of the sequential hologram data.
1. A system comprising:
a holographic capturing system comprising:
an optical system configured to generate an optical hologram of a live scene that comprises one or more three-dimensional (3D) objects; and
an optical sensor configured to capture sequential optical holograms of the live scene and output sequential hologram data associated with the sequential optical holograms of the live scene, each optical hologram being associated with respective hologram data; and
a holographic display system configured to optically reconstruct the live scene in a 3D space based on at least part of the sequential hologram data.
2. The system of claim 1 , further comprising a computing device coupled between the holographic capturing system and the holographic display system,
wherein the computing device is configured to receive the at least part of the sequential hologram data from the optical sensor and generate digital holograms associated with the live scene based on the at least part of the sequential hologram data, and
wherein the holographic display system is configured to receive the digital holograms associated with the live scene from the computing device and reconstruct the live scene in the 3D space based on the digital holograms.
3. The system of claim 2 , wherein the holographic capturing system is configured to capture the sequential optical holograms and generate the sequential hologram data, without storing the sequential optical holograms and the sequential hologram data, and
wherein the computing device is configured to process the at least part of the sequential hologram data to generate the digital holograms, without storing the at least part of the sequential hologram data and the digital holograms.
4. The system of claim 2 , wherein the holographic capturing system, the computing device, and the holographic display system are configured together to capture optical holograms of the live scene and optically reconstruct the live scene in real time.
5. The system of claim 2 , wherein a digital hologram comprises an amplitude-like hologram, and the holographic display system comprises a display for phase modulation.
6. The system of claim 2 , wherein the optical sensor comprises a digital sensor, and the sequential hologram data comprises a stream of digital data, and
wherein the digital data comprises an array of data bits.
7. The system of claim 2 , further comprising a frame grabber coupled to the optical sensor and configured to select respective hologram data of one or more optical holograms among the sequential optical holograms to be transmitted to the computing device.
8. The system of claim 7 , wherein the frame grabber comprises at least one of
a frame-buffer-based grabber configured to deposit the respective hologram data in a frame buffer of the frame grabber and subsequently into the computing device, or
a first in, first out (FIFO)-based grabber configured to deposit the respective hologram data directly into the computing device.
9. The system of claim 7 , wherein the frame grabber is externally and respectively coupled to each of the optical sensor and the computing device.
10. The system of claim 7 , wherein the frame grabber is included in the optical sensor or in the computing device.
11. The system of claim 2 , wherein the optical sensor comprises a plurality of sensing pixels in an active area of the optical sensor, and the holographic display system comprises a display having a plurality of display elements, and
wherein the computing device is configured to process the at least part of the sequential hologram data to generate the digital holograms associated with the live scene based on at least one of a pitch of the sensing pixels, a pitch of the display elements, a size of the active area of the optical sensor, or a size of the display.
12. The system of claim 11 , wherein the pitch of the sensing pixels is associated with a resolution of a captured optical hologram and a capturable size of a scene, and
wherein the pitch of the display elements is associated with an acceptable viewing angle of a reconstructed scene and the size of the display.
13. The system of claim 11 , wherein the computing device is configured to suppress a mismatch between a captured optical hologram of the live scene and a reconstruction of the live scene, and
wherein the mismatch is associated with at least one of
a difference between the pitch of the sensing pixels and the pitch of the display elements, or
a difference between the size of the active area of the optical sensor and the size of the display.
14. The system of claim 11 , wherein the computing device is configured to perform at least one of
scaling a first digital hologram associated with a captured optical hologram based on a ratio between the pitch of the sensing pixels and the pitch of the display elements, or
adjusting the scaled first digital hologram to generate a second digital hologram to be modulated on the display based on the size of the display and the size of the hologram data.
15. The system of claim 14 , wherein the computing device is configured to perform the scaling using at least one of one or more interpolation algorithms comprising linear interpolation, nearest neighbor interpolation, cubic spline interpolation, shape-preserving interpolation, Biharomic interpolation, and thin-plate spline interpolation.
16. The system of claim 11 , wherein the computing device is configured to resample a first digital hologram associated with a captured optical hologram to be a second digital hologram to be modulated on the display using Fourier transform and inverse Fourier transform, the first digital hologram being associated with the pitch of the sensing pixels, the second digital hologram being associated with the pitch of the display elements.
17. The system of claim 16 , wherein the computing device is configured to:
perform the Fourier transform on the first digital hologram to generate a transformed first digital hologram, and
if the pitch of the sensing pixels is larger than the pitch of the display elements,
perform zero-padding on the transformed first digital hologram based on a ratio between the pitch of the sensing pixels and the pitch of the display elements, then
perform the inverse Fourier transform on the transformed first digital hologram with the zero-padding to obtain the second digital hologram.
18. The system of claim 16 , wherein the computing device is configured to:
perform the Fourier transform on the first digital hologram to generate a transformed first digital hologram, and
in response to determining the pitch of the sensing pixels is smaller than the pitch of the display elements,
crop the transformed first digital hologram based on a ratio between the pitch of the sensing pixels and the pitch of the display elements, then
perform the inverse Fourier transform on the cropped transformed first digital hologram to obtain the second digital hologram.
19. The system of claim 16 , wherein the computing device is configured to resample the first digital hologram to be the second digital hologram by respectively resampling central points of the plurality of sensing pixels of the optical sensor to match centroids of the plurality of display elements of the display.
20. The system of claim 19 , wherein the plurality of sensing pixels of the optical sensor is regularly arranged in the active area of the optical sensor, and the plurality of display elements is regularly arranged in the display, and
wherein the central points of the plurality of sensing pixels are regularly spaced, and the centroids of the plurality of display elements of the display are regularly spaced.
21. The system of claim 19 , wherein the plurality of sensing pixels of the optical sensor is regularly arranged in the active area of the optical sensor, and the plurality of display elements is irregularly arranged in the display, and
wherein the central points of the plurality of sensing pixels are regularly spaced, and the centroids of the plurality of display elements of the display are irregularly spaced.
22. The system of claim 21 , wherein the computing device is configured to resample regularly spaced center points of the plurality of sensing pixels to match irregularly spaced centroids of the plurality of display elements by determining a position of each centroid of the plurality of display elements based on a weighted sum of adjacent center points around the centroid using one or more weighting algorithms.
23. The system of claim 21 , wherein the plurality of display elements forms a Voronoi pattern.
24. The system of claim 19 , wherein the plurality of sensing pixels of the optical sensor is irregularly arranged in the active area of the optical sensor, and the plurality of display elements is irregularly arranged in the display, and
wherein the central points of the plurality of sensing pixels are irregularly spaced, and the centroids of the plurality of display elements of the display are irregularly spaced.
25. The system of claim 24 , wherein an irregular pattern formed by the plurality of sensing pixels match an irregular pattern formed by the plurality of display elements.
26. The system of claim 1 , wherein the optical system comprises:
an interferometer, wherein the optical hologram comprises an interference pattern of an object beam interacting with the live scene and a reference beam interfering with the object beam by the interferometer;
a coherent light source configured to emit a coherent light beam;
a beam splitter configured to split the coherent light beam from the coherent light source into the object beam and the reference beam; and
a beam combiner, wherein the live scene is on an optical path of the object beam upstream the beam combiner, and wherein the beam combiner is configured to superimpose the reference beam and the object beam to form the interference pattern.
27. The system of claim 26 , wherein the optical sensor is arranged downstream of the beam combiner and configured to directly capture the interference pattern on an active area of the optical sensor, and
wherein there is no optical lens between the beam combiner and the optical sensor, and wherein there is no optical lens on an optical path of the reference beam between the beam splitter and the beam combiner.
28. The system of claim 1 , wherein the holographic display system comprises:
a display comprising a plurality of display elements; and
a driving device coupled to the display,
wherein the driving device is configured to:
generate control signals for the plurality of display elements of the display based on a digital hologram associated with the live scene; and
transmit the control signals to the display to modulate the plurality of display elements of the display based on the control signals.
29. The system of claim 28 , further comprising a computing device coupled between the holographic capturing system and the holographic display system,
wherein the computing device is configured to receive the at least part of the sequential hologram data from the optical sensor and generate digital holograms associated with the live scene based on the at least part of the sequential hologram data, and
wherein the holographic display system is configured to receive the digital holograms associated with the live scene from the computing device and reconstruct the live scene in the 3D space based on the digital holograms.
30. The system of claim 29 , wherein the digital holograms comprise a series of groups of digital holograms for a plurality of colors, and wherein the holographic display system further comprises an illuminator comprising a plurality of coherent light elements for the plurality of colors, and
wherein the driving device is configured to:
sequentially modulate the display with a first digital hologram for a first color during a first time period and modulate the display with a second digital hologram for a second color during a second, sequential time period; and
control the illuminator to sequentially turn on a first coherent light element to emit light with the first color during the first time period and a second coherent light element to emit light with the second color during the second, sequential time period.