Holographically displaying three-dimensional objects
Methods, apparatus, devices, subsystems, and systems for holographically displaying three-dimensional objects are provided. In one aspect, a system includes: a display having a plurality of display elements and a driving device coupled to the driving device and configured to: obtain a hologram for the display, wherein the hologram comprises, for each display element of the plurality of display elements, a respective sum of electromagnetic (EM) field contributions of a plurality of primitives corresponding to at least one object to the display element; and generate, for each display element of the plurality of display elements, a respective modulation control signal based on the respective sum of EM field contributions of the plurality of primitives to the display element.
1. A system comprising:
a display comprising a plurality of display elements;
an optical device adjacent to the display, wherein the optical device comprises a substrate and an optically diffractive element disposed on the substrate and configured to diffract light to the display; and
a driving device coupled to the display and configured to:
obtain a hologram for the display, wherein the hologram comprises, for each display element of the plurality of display elements, a respective sum of electromagnetic (EM) field contributions of a plurality of primitives corresponding to at least one object to the display element; and
generate, for each display element of the plurality of display elements, a respective modulation control signal based on the respective sum of EM field contributions of the plurality of primitives to the display element.
2. The system of claim 1 , wherein the driving device comprises:
a memory configured to store the hologram; and
a display driver coupled to the memory and the display.
3. The system of claim 1 , further comprising a processing device coupled to the driving device and configured to:
obtain information of the plurality of primitives corresponding to the at least one object, wherein the information comprises respective primitive identifiers of the plurality of primitives;
obtain primitive data of the plurality of primitives based on the information of the plurality of primitives;
for each primitive of the plurality of primitives, determine an electromagnetic (EM) field contribution to each of the plurality of display elements of the display based on primitive data of the primitive; and
for each of the plurality of display elements of the display, generate a sum of the EM field contributions of the plurality of primitives to the display element.
4. The system of claim 3 , wherein the processing device comprises:
a command processor configured to:
receive a command from a computing device, the command comprising the information of the plurality of primitives corresponding to the at least one object, and
process the command to obtain the primitive data of the plurality of primitives from the computing device based on the information of the plurality of primitives;
a plurality of computing units configured to calculate an electromagnetic (EM) field contribution of each of the plurality of primitives to each of the plurality of display elements based on the primitive data of the plurality of primitives; and
an accumulator configured to:
accumulate EM field contributions of the plurality of primitives to each of the plurality of display elements, and
generate the hologram comprising a respective sum of the EM field contributions of the plurality of primitives to each of the plurality of display elements.
5. The system of claim 1 , further comprising an illuminator,
wherein the driving device comprises an illuminator driver for the illuminator, and
wherein the illuminator driver is configured to generate and transmit an illumination control signal to the illuminator to activate the illuminator to illuminate light on the display such that the light is caused by modulated display elements of the display to form a volumetric light field corresponding to the at least one object.
6. The system of claim 1 , further comprising a computing device configured to:
generate scene data using a 3D simulation application running on the computing device, wherein the scene data comprises information of the plurality of primitives corresponding to the at least one object; and
generate primitive data of the plurality of primitives corresponding to the at least one object based on the scene data using an application programming interface (API).
7. The system of claim 6 , wherein the API is configured to adjust initial primitive data of the plurality of primitives generated from the scene data to generate the primitive data of the plurality of primitives, and
wherein the API is configured to:
for each of a plurality of vertices of the plurality of primitives, associating a respective vertex identifier of the vertex with respective vertex data of the vertex, and storing the association between the respective vertex identifier and the respective vertex data of the vertex in a memory; and
for each of the plurality of primitives, associating a respective primitive identifier of the primitive with one or more respective vertex identifiers of one or more vertices of the primitive in the memory, and storing an association between the respective primitive identifier and the one or more respective vertex identifiers for the primitive in the memory.
8. The system of claim 1 , wherein the display is an irregular display, and the plurality of display elements form an irregular pattern.
9. The system of claim 1 , further comprising an optical device adjacent to the display,
wherein the optical device comprises:
an optical guiding device configured to guide light to propagate along a first direction within the optical guiding device;
an in-coupling diffractive structure configured to diffract the light to propagate in the optical guiding device; and
one or more out-coupling diffractive structures arranged downstream the in-coupling diffractive structure along the first direction and configured to diffract at least part of the light out of the optical guiding device along a second direction different from the first direction.
10. The system of claim 9 , wherein the light has a spectral bandwidth with a peak wavelength,
wherein the in-coupling diffractive structure is configured to cause a first optical dispersion for the light, and at least one of the one or more out-coupling diffractive structures is configured to cause a second optical dispersion for the light, and
wherein the first optical dispersion and the second optical dispersion are compensated with each other, such that light diffracted out of the optical guiding device has no or little optical dispersion.
11. The system of claim 9 , wherein the light comprises multiple colors of light,
wherein, for each color of the multiple colors of light:
the in-coupling diffractive structure comprises a corresponding first diffraction grating for light of the color;
each of the one or more out-coupling diffractive structures comprises a corresponding second diffraction grating for the light of the color; and
the corresponding first diffraction grating and the corresponding second diffraction grating are configured to cause opposite dispersions having a same magnitude for the light of the color, and
wherein light diffracted by the corresponding first diffraction grating has a first beam width, and light diffracted by the corresponding second diffraction grating has a second beam width, and wherein the first beam width and the second beam width are identical.
12. The system of claim 9 , wherein the diffracted light from the in-coupling diffractive structure propagates via total internal reflection in the optical guiding device along the first direction to be sequentially incident on each of the one or more out-coupling diffractive structures along the first direction, and
wherein the one or more out-coupling diffractive structures are configured to have gradually increased diffraction efficiencies for the light along the first direction, such that diffracted light by each of the one or more out-coupling diffractive structures out of the optical guiding device has a same optical power.
13. The system of claim 9 , wherein the in-coupling diffractive structure is arranged in or on the optical guiding device, and wherein the one or more out-coupling diffractive structures comprise a plurality of out-coupling diffractive structures arranged in or on the optical guiding device along the first direction,
wherein the plurality of display elements extend along the first direction and a third direction perpendicular to the first direction and the second direction,
wherein the optical guiding device is a first optical guiding device, the in-coupling diffractive structure is a first in-coupling diffractive structure, and the plurality of out-coupling diffractive structures is a plurality of first out-coupling diffractive structures, and
wherein the system further comprises:
a second optical guiding device extending along the first direction and the third direction,
a plurality of second in-coupling diffractive structures arranged in or on the second optical guiding device along the first direction, and
multiple groups of second out-coupling diffractive structures, each group of second out-coupling diffractive structures being arranged in or on the second optical guiding device along the third direction,
wherein, for each of the plurality of second in-coupling diffractive structures,
the second in-coupling diffractive structure is configured to receive first light diffracted from a respective first out-coupling diffractive structure and diffract the first light to propagate in the second optical guiding device via TIR along the third direction to be sequentially incident on a corresponding group of second out-coupling diffractive structures, and
each of the corresponding group of second out-coupling diffractive structures is configured to receive second light reflected from the second optical guiding device and diffract the second light out of the second optical guiding device towards the display along the second direction.
14. The system of claim 13 , wherein the first in-coupling diffractive structure, the plurality of first out-coupling diffractive structures, the plurality of second in-coupling diffractive structures, and the multiple groups of second out-coupling diffractive structures are configured such that diffracted light from each of the multiple groups of second out-coupling diffractive structures along the second direction towards the display has a uniform optical power.
15. The system of claim 13 , wherein the multiple groups of second out-coupling diffractive structures are configured such that the diffracted light by each of the multiple groups of second out-coupling diffractive structures propagates to illuminate a respective portion of the display, a sum of the respective portions of the display being no smaller than an area of the display.
16. The system of claim 9 , wherein the light comprises multiple colors of light,
wherein each of the one or more out-coupling diffractive structure comprises:
multiple optically diffractive components respectively for the multiple colors of light; and
at least one of:
one or more color-selective polarizers configured to rotate a polarization state of one or more colors of the multiple colors of light such that each color of light is diffracted out by a corresponding optically diffractive component along the second direction, or
one or more reflective layers configured to totally reflect a single color of light and transmit one or more other colors of light such that each color of light is diffracted out by a corresponding optically diffractive component along the second direction.
17. The system of claim 9 , wherein the optical device further comprises an optically redirecting component,
wherein each of the one or more out-coupling diffractive structures is configured to diffract the light at an incident angle onto the display;
for the light that is incident on the display at the incident angle, the display diffracts the light; and
the optically redirecting component is configured to transmit a portion of the light diffracted by the display to provide a holographic scene and to redirect display zero order light away from the holographic scene in a three-dimensional (3D) space, the display zero order light comprising reflected light from the display.
18. The system of claim 9 , further comprising:
a linear polarizer configured to transmit light with a linear polarization state; and
an optical retarder configured to alter a polarization state of light passing through the optical retarder,
wherein the linear polarizer and the optical retarder are configured to cause ambient light coming from a first side of the linear polarizer to pass through the linear polarizer and the optical retarder to be incident on a display and deflected back from the display to pass through the optical retarder to be blocked from a second side of the linear polarizer by the linear polarizer, the second side of the linear polarizer being opposite to the first side of the linear polarizer, and wherein the optical device, the linear polarizer, and the optical retarder are configured to cause the light to be incident on the display and deflected back from the display to transmit from the second side of the linear polarizer through the linear polarizer.
19. The system of claim 1 , wherein the optically diffractive element comprises a diffraction grating with a diffraction efficiency for the light, the diffraction efficiently being lower than a predetermined threshold that is no greater than 20%.
20. The system of claim 1 , further comprising a polarization controller arranged between the optical device and the display,
wherein the polarization controller is configured to:
make the light from the optical device to be incident on the display have S polarization state, and
make returned light from the display to be incident on the optical device have P polarization state or an intermediate state between S and P polarization states, and wherein the polarization controller comprises a Faraday rotator.
21. The system of claim 1 , wherein the optical diffractive element is configured such that, when a plurality of different colors of light is incident on the optically diffractive element, the optically diffractive element separates or combines light of individual colors of the different colors while suppressing crosstalk between the different colors,
wherein the optically diffractive element comprises:
at least one optically diffractive component; and
at least one of one or more color-selective polarizers or at least one of one or more reflective layers, one or more transflective layers, or one or more transmissive layers.
22. The system of claim 1 , wherein the optical device further comprises an optically redirecting element disposed on the substrate,
wherein the display is configured to diffract a portion of the light illuminating one or more display elements,
wherein the optically redirecting element is configured to transmit the diffracted portion of the light to form a holographic scene and to redirect display zero order light away from the holographic scene in a three-dimensional (3D) space, the display zero order light comprising reflected light from the display, and
wherein the optically diffractive element is arranged on a first side of the substrate, facing a display surface of the display, and the optically redirecting element is arranged on a second side of the substrate opposite to the first side of the substrate.
23. The system of claim 1 , further comprising:
an illuminator arranged adjacent to the optical device and configured to provide a plurality of different colors of light to the optical device, wherein the illuminator comprises a plurality of light emitting elements each configured to emit a corresponding color of light; a coupling device arranged between the illuminator and the optical device and configured to receive the plurality of different colors of light from the illuminator and to output the plurality of different colors of light to the optical device, and
wherein the coupling device comprises:
a prism element between the illuminator and the optical device and configured to receive the plurality of different colors of light from an input surface of the prism element;
one or more expansion gratings adjacent an exit surface of the prism element, each of the one or more expansion gratings configured to expand a beam profile of a different corresponding color of light by a factor in at least one dimension; and
one or more reflectors downstream of the one or more expansion gratings, each of the one or more reflectors being configured to reflect a respective color of light into the optically diffractive element, wherein a tilt angle of each of the one or more reflectors is independently adjustable to cause a uniformity of diffraction from the optical device to the display.
24. The system of claim 23 , wherein the optical device is positioned facing a display surface of the display along a perpendicular direction to the display surface,
wherein the coupling device is configured to couple the plurality of different colors of light into the optically diffractive element from a side surface of the coupling device, and
wherein the coupling device is configured to couple the plurality of different colors of light into the optically diffractive element from a bottom surface or a top surface of the coupling device.
25. The system of claim 1 , wherein the system comprises a plurality of optical components comprising at least one first optical component configured to cause a positive optical dispersion for light incident on the display and at least one second optical component configured to cause a negative optical dispersion for the light, the light having a spectral bandwidth with a peak wavelength, and
wherein the positive optical dispersion and the negative optical dispersion are compensated with each other, such that a holographic scene reconstructed from the light has no or little light dispersion.
26. The system of claim 25 , wherein the at least one first optical component comprises a first diffraction grating, and the at least one second optical component comprises a second diffraction grating, and wherein the first diffraction grating and the second diffraction grating are configured to cause a same magnitude of dispersion for the light,
wherein the system is configured to perform one of:
i) causing the light with an odd number of reflections between the first diffraction grating and the second diffraction grating, wherein the first diffraction grating and the second diffraction grating are configured to cause a same dispersion for the light, or
ii) causing the light with an even number of reflections between the first diffraction grating and the second diffraction grating, wherein the first diffraction grating and the second diffraction grating are configured to cause opposite dispersions having a same magnitude for the light.
27. The system of claim 25 , wherein the positive optical dispersion caused by the at least one first optical component has a first magnitude of dispersion, and the negative optical dispersion caused by the at least one second optical component has a second magnitude of dispersion, and wherein a magnitude ratio of the second magnitude of dispersion over the first magnitude of dispersion is different from 1, and
wherein the at least one second optical component is arranged downstream the at least one first optical component along a light path of the light to be incident on the display, wherein the system is configured to change a beam width of the light from the at least one first optical component to the at least one second optical component by a width factor in one dimension, and wherein the width factor is identical to the magnitude ratio.
28. The system of claim 25 , wherein the plurality of optical components comprise:
a first group of optical components each configured to cause a first optical dispersion for a respective color of light of a plurality of colors of light, and
a second group of optical components each configured to cause a second optical dispersion for a respective color of light of the plurality of colors of light,
wherein the first group of optical components and the second group of optical components are configured to compensate optical dispersion for each of the plurality of colors of light and wherein at least one of the first group of optical components and the second group of optical components comprises a series of holographic gratings made of a same material.
29. The system of claim 25 , wherein the plurality of optical components are configured to compensate optical dispersion for a first color of light with a first spectral width, without compensation for a second color of light with a second spectral width that is narrower than the first spectral width.
30. A system comprising:
a display comprising a plurality of display elements;
an illuminator; and
a driving device comprising a display driver coupled to the display and an illuminator driver coupled to the illuminator,
wherein the display driver is configured to: obtain a hologram for the display, wherein the hologram comprises, for each display element of the plurality of display elements, a respective sum of electromagnetic (EM) field contributions of a plurality of primitives corresponding to at least one object to the display element; and generate, for each display element of the plurality of display elements, a respective modulation control signal based on the respective sum of EM field contributions of the plurality of primitives to the display element, and
wherein the illuminator driver is configured to generate and transmit an illumination control signal to the illuminator to activate the illuminator to illuminate light on the display such that the light is caused by modulated display elements of the display to form a volumetric light field corresponding to the at least one object.
31. A system comprising:
a display comprising a plurality of display elements;
a driving device coupled to the display and configured to: obtain a hologram for the display, wherein the hologram comprises, for each display element of the plurality of display elements, a respective sum of electromagnetic (EM) field contributions of a plurality of primitives corresponding to at least one object to the display element; and generate, for each display element of the plurality of display elements, a respective modulation control signal based on the respective sum of EM field contributions of the plurality of primitives to the display element; and
a processing device coupled to the driving device and configured to: obtain information of the plurality of primitives corresponding to the at least one object, wherein the information comprises respective primitive identifiers of the plurality of primitives; obtain primitive data of the plurality of primitives based on the information of the plurality of primitives; for each primitive of the plurality of primitives, determine an electromagnetic (EM) field contribution to each of the plurality of display elements of the display based on primitive data of the primitive; and for each of the plurality of display elements of the display, generate a sum of the EM field contributions of the plurality of primitives to the display element,
wherein the processing device comprises:
a command processor configured to: receive a command from a computing device, the command comprising the information of the plurality of primitives corresponding to the at least one object, and process the command to obtain the primitive data of the plurality of primitives from the computing device based on the information of the plurality of primitives;
a plurality of computing units configured to calculate an electromagnetic (EM) field contribution of each of the plurality of primitives to each of the plurality of display elements based on the primitive data of the plurality of primitives; and
an accumulator configured to: accumulate EM field contributions of the plurality of primitives to each of the plurality of display elements, and generate the hologram comprising a respective sum of the EM field contributions of the plurality of primitives to each of the plurality of display elements.
32. A system comprising:
a display comprising a plurality of display elements;
a driving device coupled to the display and configured to:
obtain a hologram for the display, wherein the hologram comprises, for each display element of the plurality of display elements, a respective sum of electromagnetic (EM) field contributions of a plurality of primitives corresponding to at least one object to the display element; and
generate, for each display element of the plurality of display elements, a respective modulation control signal based on the respective sum of EM field contributions of the plurality of primitives to the display element; and
an optical device adjacent to the display, wherein the optical device comprises:
an optical guiding device configured to guide light to propagate along a first direction within the optical guiding device;
an in-coupling diffractive structure configured to diffract the light to propagate in the optical guiding device; and
one or more out-coupling diffractive structures arranged downstream the in-coupling diffractive structure along the first direction and configured to diffract at least part of the light out of the optical guiding device along a second direction different from the first direction.
33. The system of claim 32 , wherein the light has a spectral bandwidth with a peak wavelength,
wherein the in-coupling diffractive structure is configured to cause a first optical dispersion for the light, and at least one of the one or more out-coupling diffractive structures is configured to cause a second optical dispersion for the light, and
wherein the first optical dispersion and the second optical dispersion are compensated with each other, such that light diffracted out of the optical guiding device has no or little optical dispersion.
34. The system of claim 32 , wherein the diffracted light from the in-coupling diffractive structure propagates via total internal reflection in the optical guiding device along the first direction to be sequentially incident on each of the one or more out-coupling diffractive structures along the first direction, and
wherein the one or more out-coupling diffractive structures are configured to have gradually increased diffraction efficiencies for the light along the first direction, such that diffracted light by each of the one or more out-coupling diffractive structures out of the optical guiding device has a same optical power.
35. The system of claim 32 , wherein the light comprises multiple colors of light,
wherein each of the one or more out-coupling diffractive structure comprises:
multiple optically diffractive components respectively for the multiple colors of light; and
at least one of:
one or more color-selective polarizers configured to rotate a polarization state of one or more colors of the multiple colors of light such that each color of light is diffracted out by a corresponding optically diffractive component along the second direction, or
one or more reflective layers configured to totally reflect a single color of light and transmit one or more other colors of light such that each color of light is diffracted out by a corresponding optically diffractive component along the second direction.
36. A system comprising:
a display comprising a plurality of display elements;
a driving device coupled to the display and configured to:
obtain a hologram for the display, wherein the hologram comprises, for each display element of the plurality of display elements, a respective sum of electromagnetic (EM) field contributions of a plurality of primitives corresponding to at least one object to the display element; and
generate, for each display element of the plurality of display elements, a respective modulation control signal based on the respective sum of EM field contributions of the plurality of primitives to the display element; and
a plurality of optical components comprising at least one first optical component configured to cause a positive optical dispersion for light incident on the display and at least one second optical component configured to cause a negative optical dispersion for the light, the light having a spectral bandwidth with a peak wavelength, wherein the positive optical dispersion and the negative optical dispersion are compensated with each other, such that a holographic scene reconstructed from the light has no or little light dispersion.