Granted Patent B2

US 12,374,247 · App. 18/906,928 · Granted Jul 29, 2025

Inventors: Kamran Qaderi (San Gabriel, CA); Jonathan Seamus Blackley (South Pasadena, CA); Watson Brent Boyett (Los Angeles, CA); Robin James Green (Duvall, WA); Stephen John Hart (San Juan Capistrano, CA); Robert Alan Hess (Mesa, AZ); Mark Anthony Loya (Temple City, CA); Benjamin Francis Neil (South Pasadena, CA); Jesus Manuel Caridad Ramirez (Altadena, CA); William Luke Snitzer (Los Angeles, CA)

Assignee: Pacific Light & Hologram, Inc.

G09G3/003G02F1/134309G02F1/13439G09G3/3413G09G3/36

Abstract

Claims (116)

1. A method comprising:

optically generating an optical hologram of a live scene that comprises one or more three-dimensional (3D) objects;

capturing sequential optical holograms of the live scene and generating sequential hologram data associated with the sequential optical holograms of the live scene, each optical hologram being associated with respective hologram data;

processing at least part of the sequential hologram data to generate digital holograms associated with the live scene based on at least one of a pitch of sensing pixels of an optical sensor, a pitch of display elements of a display, a size of an active area of the optical sensor, or a size of the display; and

reconstructing the live scene in a 3D space based on the at least part of the sequential hologram data, wherein reconstructing the live scene in the 3D space based on the at least part of the hologram data comprises: reconstructing the live scene in the 3D space based on the digital holograms.

2. The method of claim 1 , wherein each of the digital holograms comprises an amplitude-like hologram, and

wherein reconstructing the live scene in the 3D space based on the digital holograms comprises:

modulating a display for phase modulation directly with the digital hologram.

3. The method of claim 1 , wherein the sequential hologram data comprises a stream of digital data, and wherein the digital data comprises an array of data bits.

4. The method of claim 1 , further comprising:

selecting respective hologram data of one or more optical holograms among the sequential optical holograms by a frame grabber,

wherein processing the at least part of the sequential hologram data to generate the digital holograms associated with the live scene comprises:

generating the digital holograms associated with the live scene based on the selected respective hologram data of the one or more optical holograms.

5. The method of claim 4 , wherein the frame grabber comprises one of:

a frame-buffer-based grabber configured to deposit the respective hologram data in a frame buffer of the frame grabber before transmitting for generating the digital holograms, or

a first in, first out (FIFO)-based grabber configured to transmit the respective hologram data directly for generating the digital holograms.

6. The method of claim 1 , wherein the pitch of the sensing pixels is associated with a resolution of a captured optical hologram and a capturable size of a scene, and

wherein the pitch of the display elements is associated with an acceptable viewing angle of a reconstructed scene and the size of the display.

7. The method of claim 1 , wherein processing the at least part of the sequential hologram data to generate the digital holograms associated with the live scene comprises:

suppressing a mismatch between a captured optical hologram of the live scene and a reconstruction of the live scene, and

wherein the mismatch is associated with at least one of

a difference between the pitch of the sensing pixels and the pitch of the display elements, or

a difference between the size of the active area of the optical sensor and the size of the display.

8. The method of claim 1 , wherein processing the at least part of the sequential hologram data to generate the digital holograms associated with the live scene comprises at least one of:

scaling a first digital hologram associated with a captured optical hologram based on a ratio between the pitch of the sensing pixels and the pitch of the display elements, or

cropping the scaled first digital hologram to generate a second digital hologram to be modulated on the display based on the size of the display and the size of the hologram data.

9. The method of claim 8 , wherein scaling the size of the first digital hologram associated with the captured optical hologram comprises:

using at least one of one or more interpolation algorithms comprising linear interpolation, nearest neighbor interpolation, cubic spline interpolation, shape-preserving interpolation, Biharmonic interpolation, and thin-plate spline interpolation.

10. The method of claim 1 , wherein processing the at least part of the sequential hologram data to generate the digital holograms associated with the live scene comprises:

resampling a first digital hologram associated with a captured optical hologram to be a second digital hologram to be modulated on the display using Fourier transform and inverse Fourier transform, the first digital hologram being associated with the pitch of the sensing pixels, the second digital hologram being associated with the pitch of the display elements.

11. The method of claim 10 , wherein resampling the first digital hologram associated with the captured optical hologram to be the second digital hologram to be modulated on the display comprises:

performing the Fourier transform on the first digital hologram to generate a transformed first digital hologram, and

in response to determining the pitch of the sensing pixels is smaller than the pitch of the display elements,

cropping the transformed first digital hologram based on a ratio between the pitch of the sensing pixels and the pitch of the display element, then

performing the inverse Fourier transform on the cropped transformed first digital hologram to obtain the second digital hologram.

12. The method of claim 10 , wherein resampling the first digital hologram associated with the captured optical hologram to be the second digital hologram to be modulated on the display comprises:

performing the Fourier transform on the first digital hologram to generate a transformed first digital hologram, and

if the pitch of the sensing pixels is larger than the pitch of the display elements,

adding one or more zero-pads to the transformed first digital hologram based on a ratio between the pitch of the sensing pixels and the pitch of the display element, then

performing the inverse Fourier transform on the transformed first digital hologram with the added one or more zero-pads to obtain the second digital hologram.

13. The method of claim 10 , wherein resampling the first digital hologram associated with the captured optical hologram to be the second digital hologram to be modulated on the display comprises:

resampling the first digital hologram to be the second digital hologram by respectively resampling central points of the plurality of sensing pixels of the optical sensor to match centroids of a plurality of display elements of the display.

14. The method of claim 13 , wherein the plurality of sensing pixels of the optical sensor is regularly arranged in the active area of the optical sensor, and the plurality of display elements is regularly arranged in the display, and

wherein the central points of the plurality of sensing pixels are regularly spaced, and the centroids of the plurality of display elements of the display are regularly spaced.

15. The method of claim 13 , wherein the plurality of sensing pixels of the optical sensor is regularly arranged in the active area of the optical sensor, and the plurality of display elements is irregularly arranged in the display, and

wherein the central points of the plurality of sensing pixels are regularly spaced, and the centroids of the plurality of display elements of the display are irregularly spaced.

16. The method of claim 15 , wherein resampling the first digital hologram associated with the captured optical hologram to be the second digital hologram to be modulated on the display comprises:

resampling regularly spaced center points of the plurality of sensing pixels to match irregularly spaced centroids of the plurality of display elements by determining a position of each centroid of the plurality of display elements based on a weighted sum of adjacent center points around the centroid using one or more weighting algorithms.

17. The method of claim 13 , wherein the plurality of sensing pixels of the optical sensor is irregularly arranged in the active area of the optical sensor, and the plurality of display elements is irregularly arranged in the display,

wherein the central points of the plurality of sensing pixels are irregularly spaced, and the centroids of the plurality of display elements of the display are irregularly spaced, and wherein an irregular pattern formed by the plurality of sensing pixels match an irregular pattern formed by the plurality of display elements.

18. The method of claim 1 , wherein optically generating the optical hologram of the live scene comprises:

forming an interference pattern by interfering an object beam interacting with the live scene with a reference beam, wherein the object beam and the reference beam are coherent light beams, and the optical hologram comprises the interference pattern.

19. The method of claim 18 , wherein capturing the sequential optical holograms of the live scene comprises:

directly capturing the interference pattern on active area of an optical sensor.

20. The method of claim 19 , further comprising:

magnifying or demagnifying the object beam after interacting with the live scene to be compatible with an active area of the optical sensor.

21. The method of claim 18 , further comprising:

dynamically adjusting a phase shift of one of the object beam and the reference beam before the interference pattern is formed.

22. The method of claim 21 , wherein dynamically adjusting the phase shift of one of the object beam and the reference beam comprises: sequentially adjusting the phase shift to be a series of predetermined values in a time period, and

wherein capturing the sequential optical holograms of the live scene comprises: capturing corresponding sequential optical holograms of the live scene in the time period.

23. The method of claim 22 , further comprising:

generating a digital hologram of the live scene with noise suppression based on the corresponding sequential optical holograms, and

wherein generating the digital hologram of the live scene with noise suppression based on the corresponding sequential optical holograms comprises:

processing the corresponding sequential optical holograms to obtain corresponding raw digital holograms, and

performing one or more mathematical operations on the corresponding raw digital holograms to generate the digital hologram of the live scene.

24. The method of claim 23 , wherein the series of predetermined values for the phase shift comprises 0, pi/2, pi, 3pi/2, and the digital hologram is calculated based on an expression as follows:

final_hologram=(hologram_0-hologram_pi)/(hologram_pi/2-hologram_3pi/2), where final_hologram represents the digital hologram,

hologram_0 represents a first corresponding raw digital hologram based on a first corresponding optical hologram with 0 phase shift,

hologram pi/2 represents a second corresponding raw digital hologram based on a second corresponding optical hologram with pi/2 phase shift,

hologram pi represents a third corresponding raw digital hologram based on a third corresponding optical hologram with pi phase shift, and

hologram_3pi/2 represents a fourth corresponding raw digital hologram based on a fourth corresponding optical hologram with 3pi/2 phase shift.

25. The method of claim 1 , wherein optically generating the optical hologram of the live scene comprises:

sequentially and alternatively emitting light with a plurality of colors to sequentially and alternatively generate optical holograms for the plurality of colors,

wherein capturing the sequential optical holograms of the live scene comprises:

sequentially transmitting only light with an individual color, while blocking light with other colors.

26. The method of claim 1 , wherein optically generating the optical hologram of the live scene comprises: emitting light with a plurality of colors simultaneously, and

wherein capturing the sequential optical holograms of the live scene comprises:

capturing the optical hologram by corresponding groups of adjacent sensing pixels of a plurality of sensing pixels of an optical sensor, with a color filter array arranged on the optical sensor, wherein the color filter array comprises groups of different color filters on the plurality of sensing pixels of the optical sensor, the different color filters being associated with the plurality of colors, each group of the different color filters being arranged on a corresponding group of adjacent sensing pixels of the plurality of sensing pixels.

27. The method of claim 26 , wherein generating sequential hologram data associated with the sequential optical holograms of the live scene comprises: determining hologram data for each of the plurality of colors based on the optical hologram,

wherein the method further comprises: generating digital holograms for the plurality of colors based on the optical hologram, and

wherein reconstructing the live scene in a 3D space based on at least part of the hologram data comprises: reconstructing the live scene in the 3D space based on the digital holograms.

28. The method of claim 1 , wherein reconstructing the live scene in a 3D space based on at least part of the hologram data comprises:

generating control signals for a plurality of display elements of a display based on a digital hologram associated with the live scene; and

modulate the plurality of display elements of the display based on the control signals,

wherein the method further comprises:

generating digital holograms associated with the live scene based on the at least part of the sequential hologram data, wherein the digital holograms comprise a series of groups of digital holograms for a plurality of colors, and

wherein reconstructing the live scene in a 3D space based on at least part of the hologram data comprises:

sequentially modulating the display with a first digital hologram for a first color during a first time period and modulating the display with a second digital hologram for a second color during a second, sequential time period, and

sequentially turning on a first coherent light element to emit light with the first color during the first time period and a second coherent light element to emit light with the second color during the second, sequential time period.

29. A method comprising:

optically generating an optical hologram of a live scene that comprises one or more three-dimensional (3D) objects;

capturing sequential optical holograms of the live scene and generating sequential hologram data associated with the sequential optical holograms of the live scene, each optical hologram being associated with respective hologram data;

selecting respective hologram data of one or more optical holograms among the sequential optical holograms by a frame grabber;

processing at least part of the sequential hologram data to generate digital holograms associated with the live scene based on the selected respective hologram data of the one or more optical holograms; and

reconstructing the live scene in a 3D space based on the digital holograms.

30. The method of claim 29 , wherein the frame grabber comprises one of:

a frame-buffer-based grabber configured to deposit the respective hologram data in a frame buffer of the frame grabber before transmitting for generating the digital holograms, or

a first in, first out (FIFO)-based grabber configured to transmit the respective hologram data directly for generating the digital holograms.

31. A method comprising:

optically generating an optical hologram of a live scene that comprises one or more three-dimensional (3D) objects;

capturing sequential optical holograms of the live scene and generating sequential hologram data associated with the sequential optical holograms of the live scene, each optical hologram being associated with respective hologram data; and

reconstructing the live scene in a 3D space based on at least part of the sequential hologram data,

wherein optically generating the optical hologram of the live scene comprises: forming an interference pattern by interfering an object beam interacting with the live scene with a reference beam, wherein the object beam and the reference beam are coherent light beams, and the optical hologram comprises the interference pattern,

wherein the method further comprises: before the interference pattern is formed, dynamically adjusting a phase shift of one of the object beam and the reference beam by sequentially adjusting the phase shift to be a series of predetermined values in a time period, and

wherein capturing the sequential optical holograms of the live scene comprises: capturing corresponding sequential optical holograms of the live scene in the time period.

32. The method of claim 31 , further comprising:

generating a digital hologram of the live scene with noise suppression based on the corresponding sequential optical holograms, and

wherein generating the digital hologram of the live scene with noise suppression based on the corresponding sequential optical holograms comprises:

processing the corresponding sequential optical holograms to obtain corresponding raw digital holograms, and

performing one or more mathematical operations on the corresponding raw digital holograms to generate the digital hologram of the live scene.

33. The method of claim 32 , wherein the series of predetermined values for the phase shift comprises 0, pi/2, pi, 3pi/2, and the digital hologram is calculated based on an expression as follows:

final_hologram=(hologram_0-hologram pi)/(hologram_pi/2-hologram_3pi/2), where final_hologram represents the digital hologram,

hologram_0 represents a first corresponding raw digital hologram based on a first corresponding optical hologram with 0 phase shift,

hologram_pi/2 represents a second corresponding raw digital hologram based on a second corresponding optical hologram with pi/2 phase shift,

hologram pi represents a third corresponding raw digital hologram based on a third corresponding optical hologram with pi phase shift, and

hologram_3pi/2 represents a fourth corresponding raw digital hologram based on a fourth corresponding optical hologram with 3pi/2 phase shift.

Assignments (1)

Continuity (6)

References Cited (307)

Cited By (2)

US 12,561,907 US 12,579,919