Active illumination systems
Fiducial patterns that produce diffraction patterns at a camera sensor are etched or otherwise provided on the surface of a cover glass (CG) in front of a camera. An active light source injects light into the cover glass or into a diffractive optical element to strengthen the signal from the fiducial pattern, or alternatively an active light source is used to reflect light off a reflective fiducial pattern on the cover glass, thus requiring fewer frames to capture and process the diffraction pattern caused by the fiducial pattern.
1 . A system, comprising:
a camera comprising a camera lens and an image sensor;
a transparent element on an object side of the camera lens, the transparent element including a fiducial pattern on a camera-facing surface configured to cause a diffraction pattern in images formed by the camera lens at a surface of the image sensor;
a light source configured to illuminate the fiducial pattern to strengthen the signal from the fiducial pattern at the image sensor;
wherein the camera is configured to capture light passing through the transparent element and fiducial pattern distinct from light from the light source; and
one or more processors configured to process one or more images including the diffraction pattern captured by the camera to extract the fiducial pattern.
2 . The system as recited in claim 1 , further comprising a prism attached to the transparent element, wherein the light source is configured to inject light into the prism, and wherein the prism is configured to redirect the light into the transparent element at an entrance pupil.
3 . The system as recited in claim 2 , wherein the transparent element is configured to propagate the injected light using total internal reflection (TIR), wherein the fiducial pattern is on a camera-facing surface of the transparent element and comprises a plurality of markers, and wherein a portion of the propagated light exits the transparent element at the markers of the fiducial pattern on the transparent element towards the camera lens.
4 . The system as recited in claim 3 , wherein the markers of the fiducial pattern are formed on the surface of the transparent element by laser etching or are formed of a diffuser film on the surface of the transparent element.
5 . The system as recited in claim 1 , wherein the transparent element comprises:
a diffractive optical element;
an in-couple grating corresponding to an entrance pupil; and
an out-couple grating corresponding to an exit pupil, wherein the fiducial pattern is formed on the transparent element at the exit pupil;
wherein the light source is configured to inject light at the in-couple grating;
wherein the diffractive optical element is configured to transmit the light to the out-couple grating; and
wherein the transmitted light exits the transparent element at the exit pupil towards the camera lens to illuminate the diffraction pattern.
6 . The system as recited in claim 1 , wherein the light source is configured to emit light towards the fiducial pattern on the surface of the transparent element, and wherein the fiducial pattern is configured to reflect at least a portion of the light back towards the camera lens.
7 . The system as recited in claim 6 , wherein the fiducial pattern is formed of a reflective or retroreflective material on the surface of the transparent element.
8 . The system as recited in claim 1 , wherein the light source is a synchronized light source that is pulsed to match exposure time of the camera.
9 . The system as recited in claim 1 , wherein the light source is a light-emitting diode (LED) light source or a vertical cavity surface emitting laser (VCSEL) light source.
10 . The system as recited in claim 1 , wherein the light source is a visible light, infrared (IR) light, or near-infrared (NIR) light source, and wherein the camera is configured to capture light in a corresponding wavelength of the light source.
11 . The system as recited in claim 1 , wherein the one or more processors are further configured to:
determine offsets of the transparent element with respect to the camera lens from the extracted diffraction pattern; and
apply the determined offsets to one or more images captured by the camera during processing of the one or more images to account for distortion in the one or more images caused by a corresponding shift in the transparent element with respect to the camera lens.
12 . The system as recited in claim 1 , wherein the one or more processors are further configured to detect defects on the transparent element from the extracted diffraction pattern.
13 . The system as recited in claim 1 , wherein the one or more processors are further configured to detect intrinsic changes in the camera from the extracted diffraction pattern.
14 . The system as recited in claim 1 , wherein the transparent element is a cover glass, and wherein the camera, the light source, and the cover glass are components of a head-mounted device (HMD).
15 . The system as recited in claim 1 , wherein:
the fiducial pattern is formed on a surface of the transparent element using a pad print and laser ablation process, using a laser surface marking process, or using a nano-imprinting lithography process, or
the fiducial pattern is formed on a film using a nano-imprinting lithography process, and wherein the film is laminated onto a surface of the transparent element.
16 . A method, comprising:
illuminating, by a light source, a fiducial pattern on a surface of a transparent element on an object side of a camera lens;
causing, by the illuminated fiducial pattern, a diffraction pattern in images formed by the camera lens at a surface of an image sensor;
capturing, by the image sensor, one or more images including the diffraction pattern and based on light passing through the transparent element and fiducial pattern distinct from light from the light source; and
processing, by one or more processors, one or more images including the diffraction pattern captured by the camera to extract the fiducial pattern;
wherein the light source illuminating the fiducial pattern strengthens the signal from the fiducial pattern at the image sensor.
17 . The method as recited in claim 16 , wherein the fiducial pattern is on a camera-facing surface of the transparent element and comprises a plurality of markers, wherein illuminating the fiducial pattern on the surface of the transparent element comprises:
injecting, by the light source, the light into a prism attached to the transparent element;
redirecting, by the prism, the light into the transparent element at an entrance pupil; and
propagating, by the transparent element, the injected light using total internal reflection (TIR), wherein a portion of the propagated light exits the transparent element at the markers of the fiducial pattern on the transparent element towards the camera lens.
18 . The method as recited in claim 16 , wherein illuminating the fiducial pattern on the surface of the transparent element comprises:
injecting, by the light source, the light into an in-couple grating corresponding to an entrance pupil; and
transmitting, by a diffractive optical element the light to an out-couple grating corresponding to an exit pupil;
wherein the transmitted light exits the transparent element at the exit pupil towards the camera lens to illuminate the diffraction pattern.
19 . The method as recited in claim 16 , wherein illuminating the fiducial pattern on the surface of the transparent element comprises emitting, by the light source, light towards the fiducial pattern on the surface of the transparent element, wherein the fiducial pattern is formed of a reflective or retroreflective material on the surface of the transparent element and is configured to reflect at least a portion of the light back towards the camera lens.
20 . The method as recited in claim 16 , wherein the light source is a visible light, infrared (IR) light, or near-infrared (NIR) light source, and wherein the camera is configured to capture light in a corresponding wavelength of the light source, the method further comprising:
determining, by the one or more processors, a shift of the transparent element with respect to the camera lens from the extracted pattern; and
adjusting processing of one or more additional images captured by the camera to account for the determined shift in the transparent element with respect to the camera lens.