IP Library Granted Patent US 10,708,524
Granted Patent B2
US 10,708,524 · App. 16/042,167 · Granted Jul 7, 2020

Methods and apparatus for high dynamic range imaging

Inventors: Marko Mlinar (Horjul, SI); Simon Srot (Ljubljana, SI)
Assignee: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
H04N5/35581H04N1/64H04N5/2355H04N5/2356H04N9/07B60R21/01538
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,708,524
App. No.
16/042,167
Granted
Jul 7, 2020
Kind
B2
Abstract

Various embodiments of the present technology comprise a method and apparatus for high dynamic range imaging. According to various embodiments, the method and apparatus for high dynamic range imaging is configured to select the “best” signal for each pixel location to construct an HDR output. In one embodiment, the “best” signal is select among various pixel signals and the pixel signal selected as the “best” signal is based on the value of the pixel signals and identification of non-saturated. If only one pixel value for a particular pixel location is non-saturated, then the “best” signal is the non-saturated signal. If more than one pixel value is non-saturated, then the “best” signal is the pixel signal with the greatest value. If two more pixel values are non-saturated and both have equally great values, then the “best” signal is the pixel signal with the shortest integration time.

Claims (70)

1. An imaging device, comprising:

a pixel array comprising a plurality of pixels, and configured to generate:

a first image frame comprising a plurality of first pixel signals with a first integration time; and

a second image frame comprising a plurality of second pixel signals with a second integration time; wherein the first integration time is different from the second integration time; and

an image signal processor connected to the pixel array and configured to:

determine a first value of a first pixel signal from the plurality of first pixel signals;

determine a second value of a second pixel signal from the plurality of second pixel signals; wherein the first pixel signal and the second pixel signal are generated from a same pixel from the plurality of pixels;

identify the first pixel signal as one of: saturated and non-saturated according to the first value;

identify the second pixel signal as one of: saturated and non-saturated according to the second value;

select a best signal among the first pixel signal and the second pixel signal; wherein the best signal is:

non-saturated if at least one pixel signal is non-saturated; and

saturated only if both the first and second pixel signals are saturated; and

construct a high dynamic range (HDR) output utilizing the best signal.

2. The imaging device of claim 1 , wherein, if both the first pixel signal and the second pixel signal are non-saturated, then the best signal is the pixel signal with the greatest value.

3. The imaging device of claim 1 , wherein, if both the first pixel signal and the second pixel signal are non-saturated and both have the same value, then the best signal is the pixel signal with the shortest integration time.

4. The imaging device of claim 1 , wherein if both the first pixel signal and the second pixel signal are saturated, then the best pixel signal is the pixel signal with the shortest integration time.

5. The imaging device of claim 1 , wherein the image signal processor is further configured to apply a first linearization gain to the plurality of first pixel signals and apply a second linearization gain to the plurality of second pixel signals.

6. The imaging device of claim 1 , wherein the image signal processor is further configured to compand the best signal from an N-bit value to an M-bit value, where N is greater than M, prior to constructing the HDR output.

7. The imaging device of claim 1 , wherein the image signal processor is further configured to assign at least a 1-bit code corresponding to the first integration time to the first pixel signal and assign at least a 1-bit code corresponding to the second integration time to the second pixel signal.

8. A method for generating a high dynamic range image, comprising:

generating a plurality of sequential image frames with a pixel array,

wherein:

the pixel array comprises a plurality of pixels; and

each pixel is defined by a pixel location; and

wherein:

each image frame comprises a plurality of pixel signals;

each image frame has a different integration time; and

each pixel signal corresponds to one pixel from the pixel array;

determining a value for each pixel signal from each image frame;

determining, for each pixel signal, if the pixel signal is one of: non-saturated and saturated;

for each pixel location, selecting a best signal among the plurality of image frames;

wherein the best signal is:

non-saturated if at least one pixel signal is non-saturated; and

saturated only if all of the pixel signals are saturated; and

constructing a high dynamic range (HDR) output utilizing the best pixel signal.

9. The method of claim 8 , wherein, if all pixel signals for one pixel location are non-saturated, then select the pixel signal with the greatest value.

10. The method of claim 8 , wherein, if all pixel signals for one pixel location are non-saturated and all pixel signals have the same value, then select the pixel signal with the shortest integration time.

11. The method of claim 8 , wherein, if all of the pixel signals for one pixel location are saturated, then select the pixel signal with the shortest integration time.

12. The method of claim 8 , further comprising applying a linearization gain to each pixel signal from each image frame, wherein the linearization gain is a ratio of a longest integration time to a shorter integration time.

13. A machine vision system, comprising:

an imaging device, comprising:

a pixel array comprising a plurality of pixels, wherein each pixel from the plurality of pixels is defined by a pixel location; and

wherein:

the pixel array is configured to generate a plurality of sequential image frames with each frame having a different integration time;

each image frame comprises a plurality of pixel signals; and

each pixel signal corresponds to one pixel from the pixel array;

an image signal processor connected to the pixel array and configured to:

determine a value for each pixel signal from each image frame;

determine, for each pixel signal, if the pixel signal is one of non-saturated and saturated according to the determined value;

for each pixel location, select a best pixel signal from the plurality of image frames;

wherein the best signal is:

 non-saturated if at least one pixel signal is non-saturated; and

 saturated only if all of the pixel signals are saturated; and

construct a high dynamic range (HDR) output utilizing the selected pixel signals; and

a host processor connected to the imaging device and configured to receive the HDR output and generate a decision according to the HDR output.

14. The machine vision system of claim 13 , wherein, if all pixel signals for one pixel location are non-saturated, then the image signal processor selects the pixel signal with the greatest value.

15. The machine vision system of claim 13 , wherein, if all pixel signals for one pixel location are non-saturated and all pixel signals have the same value, then the image signal processor selects the pixel signal with the shortest integration time.

16. The machine vision system of claim 13 , wherein if all pixel signals for one location are saturated, then the image signal processor selects the pixel signal with the shortest integration time.

17. The machine vision system of claim 13 , wherein:

the image signal processor is further configured to apply a linearization gain to each pixel signal from the plurality of image frames according to the integration time; and

the linearization gain is a ratio of a longest integration time to a shorter integration time.

18. The machine vision system of claim 13 , wherein:

the image signal processor comprises a register to store a preferred image frame index, where the preferred image frame index corresponds to one image frame from the plurality of image frames; and

the image signal processor is further configured to compare the pixel signals from the image frame having the preferred image frame index with a first predetermined threshold and a second predetermined threshold.

19. The machine vision system of claim 18 , wherein selecting one pixel signal from the plurality of image frames comprises selecting one of:

the pixel signal from a lower-index image frame if the pixel signal from the image frame having the preferred image frame index is less than the first predetermined threshold; and

the pixel signal from a higher-index image frame if the pixel signal from the image frame having the preferred image frame index is greater than or equal to the second predetermined threshold.

20. The machine vision system of claim 13 , wherein the image signal processor further performs a confidence assessment of the HDR output according to the integration time; and wherein:

the confidence assessment is HIGH if all of the selected pixel signals are from the same image frame; and

the confidence assessment is LOW if all of the selected pixel signals are from different image frames.

Assignments (3)
RELEASE OF SECURITY INTEREST IN PATENTS RECORDED AT REEL 047399, FRAME 0631 Recorded Jun 23, 2023
From: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC; FAIRCHILD SEMICONDUCTOR CORPORATION
Reel/Frame 064078/0001 →
SECURITY INTEREST Recorded Nov 1, 2018
From: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC; FAIRCHILD SEMICONDUCTOR CORPORATION
To: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
Reel/Frame 047399/0631 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 23, 2018
From: MLINAR, MARKO; SROT, SIMON
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
Reel/Frame 046427/0191 →