IP Library Granted Patent US 11,936,973
Granted Patent B2
US 11,936,973 · App. 17/732,219 · Granted Mar 19, 2024

Pulse chain-driven infrared imaging assembly

Inventors: Arjen Gerben Van der Sijde (Eindhoven, NL); Erno Fancsali (Heusden-Zolder, BE); Nicola Bettina Pfeffer (Eindhoven, NL); Pieter Johannes Quintus van Voorst Vader (Son, NL)
Assignee: Lumileds LLC
H04N23/56G01B11/22G06T7/50H04N23/71H05B45/32G06T2207/10048
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,936,973
App. No.
17/732,219
Granted
Mar 19, 2024
Kind
B2
Abstract

The invention describes an infrared imaging assembly ( 1 ) for capturing an infrared image (M 0 , M 1 ) of a scene (S), comprising an infrared-sensitive image sensor ( 14 ); an irradiator ( 10 ) comprising an array of individually addressable infrared-emitting LEDs, wherein each infrared-emitting LED is arranged to illuminate a scene region (S 1 , . . . , S 9 ); a driver ( 11 ) configured to actuate the infrared irradiator ( 10 ) by applying a switching pulse train (T 1 , . . . , T 9 ) to each infrared-emitting LED; an image analysis module ( 13 ) configured to analyse a preliminary infrared image (M 0 ) to determine the required exposure levels ( 130 ) for each of a plurality of image regions (R 1 , . . . , R 9 ); and a pulse train adjusting unit ( 12 ) configured to adjust the duration (L 1 , . . . , L 9 ) of a switching pulse train (T 1 , . . . , T 9 ) according to the required exposure levels ( 130 ). The invention also described a method of generating a depth map (D) for a scene (S); a depth map generator comprising an embodiment of the inventive infrared imaging assembly ( 1 ); and a camera comprising such a depth map generator ( 2 ).

Claims (49)

1. An infrared imaging assembly to capture an infrared image of a scene, the infrared imaging assembly comprising:

an infrared-sensitive image sensor;

an irradiator comprising an array of individually addressable infrared-emitting LEDs, each infrared-emitting LED arranged to illuminate at least one image region of a plurality of image regions of the scene;

a driver configured to actuate the irradiator by applying a switching pulse train to each infrared-emitting LED;

an image analyzer configured to analyze a preliminary image of the scene irradiated by the irradiator to determine exposure levels for each of the plurality of image regions, the driver configured to use pulses of a preliminary switching pulse train to drive each infrared-emitting LED during a scan sequence in which the preliminary image is captured; and

an adjusting unit configured to adjust driving of each infrared-emitting LED based on the exposure levels to obtain a final image, the preliminary switching pulse train having a shorter duration than a final switching pulse train used to drive the infrared-emitting LED to obtain the final image, the pulses of the preliminary switching pulse train having a shorter pulse width than pulses of the final switching pulse train.

2. The infrared imaging assembly according to claim 1 , wherein the preliminary image of the scene is obtained from a previous frame and analysis and driving of each of the individually addressable infrared-emitting LEDs is configured to be adjusted during a current frame.

3. The infrared imaging assembly according to claim 1 , wherein:

each infrared-emitting LED is arranged to illuminate a different image region of the plurality of image regions, and

the image analyzer is configured to determine the exposure level for each image region by determination of an average brightness of the image region and comparison of the average brightness to a predefined threshold.

4. The infrared imaging assembly according to claim 1 , wherein the adjusting unit is configured to adjust a duration of the final switching pulse train used to drive each of the individually addressable infrared-emitting LEDs based on the exposure levels.

5. The infrared imaging assembly according to claim 1 , wherein the driver is configured to adjust a number of pulses of the final switching pulse train to each of the individually addressable infrared-emitting LEDs, a duty cycle of the final switching pulse train having a value of a 50%.

6. The infrared imaging assembly according to claim 1 , wherein a maximum duration of the final switching pulse train to each of the individually addressable infrared-emitting LEDs is determined by a sensor integration time of the infrared-sensitive image sensor.

7. The infrared imaging assembly according to claim 1 , wherein a duration of the final switching pulse train to each of the individually addressable infrared-emitting LEDs is determined by both an amount of illumination of the image region associated with infrared-emitting LED and distance of the image region from the irradiator.

8. The infrared imaging assembly according to claim 1 , wherein a duration of the preliminary switching pulse train is limited to at most 2% of a sensor integration time of the infrared-sensitive image sensor.

9. The infrared imaging assembly according to claim 1 , wherein the driver is configured to adjust a same parameter of the final switching pulse train for each of a limited number of the individually addressable infrared-emitting LEDs, the parameter able to be adjusted differently for each of the limited number of the individually addressable infrared-emitting LEDs.

10. The infrared imaging assembly according to claim 1 , wherein the adjusting unit is configured to adjust the final switching pulse train applied to each of the individually addressable infrared-emitting LEDs based on pulse shape distortion caused by a temperature dependency of each of the individually addressable infrared-emitting LEDs.

11. The infrared imaging assembly according to claim 1 , wherein each pixel of the image sensor:

is to be read sequentially four times during capture of a single image, and

comprises four phase detectors for demodulation at using four demodulation channels of phase shifts of about 0°, about 90°, about 180°, and about 270°.

12. The infrared imaging assembly according to claim 11 , wherein the image analyzer is configured to determine a distance between the infrared-sensitive image sensor and the scene dependent on a combined phase shift that combines signal strengths at each of the phase shifts of the demodulation channels.

13. The infrared imaging assembly according to claim 1 , wherein the driver is configured to drive each infrared-emitting LED an identical number of times using a single set of pulses during the scan sequence.

14. A camera comprising:

an infrared imaging assembly comprising:

an infrared-sensitive image sensor;

an irradiator comprising an array of individually addressable infrared-emitting LEDs, each infrared-emitting LED arranged to illuminate at least one image region of a plurality of image regions of a scene;

a driver configured to actuate the irradiator by applying a switching pulse train to each infrared-emitting LED;

an image analyzer configured to analyze, during a current frame, a preliminary image of the scene irradiated by the irradiator during a previous frame to determine exposure levels for each of the plurality of image regions, the driver configured to use pulses of a preliminary switching pulse train to drive each infrared-emitting LED during a scan sequence in which the preliminary image is captured; and

an adjusting unit configured to adjust driving during the current frame of each infrared-emitting LED based on the exposure levels to obtain a final image, the preliminary switching pulse train having a shorter duration than a final switching pulse train used to drive the infrared-emitting LED to obtain the final image, the pulses of the preliminary switching pulse train having a shorter pulse width than pulses of the final switching pulse train.

15. The camera according to claim 14 , wherein:

the camera is a time-of-flight (TOF) camera configured to apply continuous-wave modulation to generate images during the previous and current frame, and

the camera further comprises a depth map generator configured to use each of the plurality of image regions, illuminated using the preliminary switching pulse train of approximately equal duration to drive each of the plurality of image regions during the preliminary image, to determine adjustment of the switching pulse train to each of the plurality of image regions during the current frame and computation of a depth map from the current frame based on TOF information of each of the plurality of image regions.

16. The camera according to claim 15 , wherein:

each pixel of the image sensor is to be read sequentially four times during capture of a single image,

each pixel of the image sensor comprises four phase detectors for demodulation at using four demodulation channels of phase shifts of about 0°, about 90°, about 180°, and about 270°, and

a phase shift of reflected light that reaches the image sensor for each pixel is used to determine the TOF information.

17. The camera according to claim 15 , wherein the image analyzer is configured to determine a distance between the infrared-sensitive image sensor and the scene dependent on a combined phase shift that combines signal strengths at a phase shift of each pixel.

18. The camera according to claim 15 , wherein:

a duration of the preliminary switching pulse train is limited to at most 2% of a sensor integration time of the infrared-sensitive image sensor, and

components of the camera are synchronized by the driver.

19. A controller comprising:

at least one processor configured to:

receive sensor signals from an infrared-sensitive image sensor, the sensor signals generated by illumination of a plurality of image regions of a scene illuminated by an array of individually addressable infrared-emitting LEDs;

analyze, during a current frame, a preliminary image of the scene illuminated during a previous frame to determine exposure levels for each of the plurality of image regions, the preliminary image obtained using pulses of a preliminary switching pulse train to drive each infrared-emitting LED during a scan sequence in which the preliminary image is captured; and

generate, for transmission to an adjusting unit, adjustment signals to adjust driving during the current frame of each of the individually addressable infrared-emitting LEDs based on the exposure levels to obtain a final image, the preliminary switching pulse train having a shorter duration than a final switching pulse train used to drive the infrared-emitting LED to obtain the final image, the pulses of the preliminary switching pulse train having a shorter pulse width than pulses of the final switching pulse train.

20. The controller according to claim 19 , wherein the at least one processor is further configured to:

use each of the plurality of image regions, illuminated using switching pulse trains of approximately equal duration during the preliminary image, to determine adjustment of a previously applied switching pulse train to each of the plurality of image regions during the current frame and computation of a depth map from the current frame based on TOF information of each of the plurality of image regions;

receive sequential information from each pixel of the image sensor during capture of a single image to determine time of flight information, each sequential information based on a different phase shift, the phase shifts being about 0°, about 90°, about 180°, and about 270°; and

determine a distance between the infrared-sensitive image sensor and the scene dependent on a combined phase shift that combines signal strengths at the phase shifts of each pixel.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 10, 2025
From: LUMILEDS LLC
To: LUMILEDS SINGAPORE PTE. LTD.
Reel/Frame 071888/0086 →
RELEASE OF SECURITY INTEREST Recorded Jan 29, 2025
From: SOUND POINT AGENCY LLC
To: LUMILEDS LLC; LUMILEDS HOLDING B.V.
Reel/Frame 070046/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2023
From: VAN DER SIJDE, ARJEN GERBEN; FANCSALI, ERNO; BETTINA, NICOLA BETTINA; VAN VOORST VADER, PIETER JOHANNES QUINTUS
To: LUMILEDS HOLDING B.V.
Reel/Frame 065838/0796 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 12, 2023
From: LUMILEDS HOLDING B.V.
To: LUMILEDS LLC
Reel/Frame 065838/0908 →
SECURITY INTEREST Recorded Jan 5, 2023
From: LUMILEDS LLC; LUMILEDS HOLDING B.V.
To: SOUND POINT AGENCY LLC
Reel/Frame 062299/0338 →