Display system for rendering a scene with multiple focal planes
A display system includes an optical element positioned in the optical path to receive light from the display module and direct the light into a field of view to display a scene. The optical element has a focal length on an optical axis of the optical element that varies in response to a control signal. A control system is programmed to generate the control signal to cause the optical element to cycle through a series of focal lengths at a frequency. Each focal length of the series corresponds to a focal plane of a series of focal planes. The control system is programmed to vary the light field from the display module to display the scene as a sequence of different image frames, the different image frames being synchronized so that each of the different image frames is presented at a corresponding one of the series of focal planes.
1 . A display system comprising:
a display element configured to provide a variable light field along an optical path of the display system;
an optical element positioned in the optical path to receive light from the display element and direct the light into a field of view of the display system to display a scene in the field of view, the optical element having a focal length on an optical axis of the optical element that varies in response to a control signal;
a control system in communication with the display element and the optical element, the control system being programmed to generate the control signal to continuously track the focal length of the optical element and cause the optical element to cycle through a series of focal lengths at a frequency, each focal length of the series corresponding to a focal plane of a series of focal planes, wherein tracking the focal plane of the optical element comprises:
obtaining a measurement of the focal length of the optical element; and
varying the light field from the display element, based on the measurement, to display the scene as a sequence of different image frames, the different image frames being synchronized with the control signal so that each of the different image frames is triggered to be presented at a focal plane of the series of focal planes;
wherein, during measuring of the focal length, the control signal is configured to cause the optical element to continuously adjust the focal length to sweep across a range between a minimum focal length and a maximum focal length.
2 . The display system of claim 1 , further comprising:
a focal length tracking element comprising a light emitting device and a light sensitive device, the focal length tracking element configured to perform the continuous tracking of the optical element by monitoring variations of the focal length of the optical element and providing continuous feedback to the control system.
3 . The display system of claim 2 , wherein
the light emitting device is configured to emit a beam of light through the optical element; and
the light sensitive device comprises a light-sensitive sensor configured to detect a deflection of the beam of light through the optical element, wherein an amount of the deflection of the beam of light is a function of the focal length of the optical element.
4 . The display system of claim 2 , wherein the light emitting device is configured to illuminate the optical element to generate an optical wavefront; and
the light sensitive device is configured to receive the optical wavefront from the optical element and determine a curvature of the optical element.
5 . The display system of claim 1 , wherein the series of focal planes comprises at least five focal planes, and wherein each image frame is presented at each of the five focal planes.
6 . The display system of claim 1 , wherein the series of focal planes comprises at least 25 focal planes, and wherein each image frame is presented at each of the 25 focal planes.
7 . The display system of claim 1 , wherein each scene frame of the display element comprises an image frame at each focal plane of the series of focal planes.
8 . The display system of claim 1 , where the display element comprises projection optics arranged to direct light from a spatial light modulator to the optical element.
9 . The display system of claim 1 , wherein the optical element comprises a focus-tunable lens.
10 . The display system of claim 1 , wherein the optical element is an adjustable mirror or an adjustable phase mask.
11 . The display system of claim 1 , wherein the display element comprises a digital micromirror array.
12 . The display system of claim 1 , wherein the display element comprises one or more of an OLED screen and a microLED screen.
13 . The display system of claim 1 , wherein presenting each of the different image frames at a corresponding plurality of the series of focal planes comprises displaying at least one pixel at more than one focal plane of the series of focal planes for a particular image frame based on a bit-depth associated with the at least one pixel, wherein the display element is configured to display the particular image frame at an intensity based on the bit-depth.
14 . The display system of claim 1 , wherein the control signal is configured to cause the optical element to continuously adjust the focal length to sweep across the range between the minimum focal length and the maximum focal length based on continuously adjusting a voltage of the control signal.
15 . A method of displaying a scene using a display device, comprising:
varying a light field projected from a display element according to a sequence of different image frames;
directing, with an optical element, the light field from the display element into a field of view of the display device; and
varying a focal length of the optical element through a series of different focal lengths while measuring the focal length of the optical element and varying the light field, based on the measuring, to display the scene by presenting each of the image frames at a plurality of the focal lengths;
wherein, during measuring of the focal length, a control signal is configured to cause the optical element to continuously adjust the focal length to sweep across a range between a minimum focal length and a maximum focal length.
16 . A method of displaying a scene comprising a series of image frames at different positions in a depth of field of the scene using a display device, comprising:
determining a display frame rate for displaying the image frames with a display element;
determining a set of focal planes for displaying a corresponding image frame of the scene;
generating a first control signal configured to cause an optical element to cycle through a range of focal lengths at a frequency lower than the display frame rate;
measuring a focal length of the optical element as the optical element cycles through the range of focal lengths; and
generating, based on the measurement, a second control signal configured to cause the display element to display the corresponding image frame when a focal length of the optical element corresponds to a plurality of focal planes of the set of focal planes, the image frame corresponding to the plurality of focal planes for the depth of field of the scene;
wherein, during measuring of the focal length, the first control signal is configured to cause the optical element to continuously adjust the focal length to sweep across a range between a minimum focal length and a maximum focal length.
17 . The method of claim 16 , further comprising:
receiving, from a sensing element, a signal representing a variation of a focal length of the optical element; and
adjusting, in response to the receiving, either the first control signal, the second control signal, or both the first and second control signals.
18 . The method of claim 16 , wherein the second control signal comprises an indicator representing, for one or more pixels of the scene, the image frame for displaying the one or more pixels by the display element.
19 . The method of claim 16 , wherein determining the frame rate comprises determining a per-frame bit-depth of the display element.
20 . The method of claim 19 , wherein determining the set of focal planes comprises selecting a maximum focal plane corresponding to a maximum distance, selecting a minimum focal plane corresponding to a minimum distance, and selecting a number of focal planes to interpolate between the maximum focal plane and the minimum focal plane based on the per-frame bit-depth of the display element.
21 . The method of claim 16 , wherein the set of focal planes comprises at least five focal planes.