IP Library Granted Patent US 8,346,008
Granted Patent B2
US 8,346,008 · App. 12/480,970 · Granted Jan 1, 2013

Systems and methods for noise reduction in high dynamic range imaging

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,346,008
App. No.
12/480,970
Granted
Jan 1, 2013
Kind
B2
Abstract

This is generally directed to systems and methods for noise reduction in high dynamic range (“HDR”) imaging systems. In some embodiments, multiple images of the same scene can be captured, where each of the images is exposed for a different amount of time. An HDR image may be created by suitably combining the images. However, the signal-to-noise ratio (“SNR”) curve of the resulting HDR image can have discontinuities in sections of the SNR curve corresponding to shifts between different exposure times. Accordingly, in some embodiments, a noise model for the HDR image can be created that takes into account these discontinuities in the SNR curve. For example, a noise model can be created that smoothes the discontinuities of the SNR curve into a continuous function. This noise model may then be used with a Bayer Filter or any other suitable noise filter to remove noise from the HDR image.

Claims (47)

1. A method of creating a noise model comprising:

determining, by control circuitry of an imaging device, a first noise model for a first light intensity period, wherein the first light intensity period is associated with a first exposure of a multiple-exposure high dynamic range image (“HDR image”);

determining, by the control circuitry, a second noise model for a second light intensity period, wherein the second light intensity period is associated with a second exposure of the HDR image;

determining, by the control circuitry, a third noise model for a first transition period comprising light intensity values between the first and second light intensity periods, wherein the third noise model is based on a ratio of attributes of at least the first and second exposures; and

filtering, by the control circuitry, noise from the HDR image based on at least the first noise model, the second noise model, and the third noise model.

2. The method of claim 1 , wherein the first exposure is taken with a first exposure time, the second exposure is taken with a second exposure time, and wherein the ratio of attributes comprises a ratio of the first exposure time to the second exposure time.

3. The method of claim 1 , wherein the third noise model is defined by a linear function.

4. The method of claim 1 , wherein the third noise model smoothes a discontinuity of a signal-to-noise ratio (“SNR”) curve of the HDR image and wherein the discontinuity is associated with a transition region between an SNR curve of the first exposure and an SNR curve of the second exposure.

5. The method of claim 1 , wherein the first noise model is described by the equation: σ i =max(σ 0 ,K·√{square root over (G a ·I i )}), wherein σ 0 is a noise floor, K is a noise model coefficient, G a is an analog gain of the imaging device, and I i is an image signal level of the HDR image.

6. The method of claim 1 , wherein the second noise model is described by the equation: σ i =R·σ 0 +K·√{square root over (R·G a ·I i )}, wherein R is a ratio of exposure times of the first exposure and the second exposure, σ 0 is a noise floor, K is a noise model coefficient, G a is an analog gain of the imaging device, and I i is an image signal level of the HDR image.

7. The method of claim 1 , further comprising:

determining, by the control circuitry, a fourth noise model for a third light intensity period, wherein the third light intensity period is associated with a third exposure of the HDR image; and

determining, by the control circuitry, a fifth noise model for a second transition period comprising light intensity values between the second and third light intensity periods, wherein the fifth noise model is associated with attributes of at least the first, second, and third exposure.

8. A method comprising:

determining, by control circuitry of a high dynamic range (“HDR”) imaging system, a start transition point on a signal-to-noise ratio (“SNR”) curve of the HDR imaging system, wherein the HDR imaging system is operable to generate an HDR image from a plurality of exposures;

determining, by the control circuitry, an end transition point on the SNR curve, wherein the start transition point and the end transition point enclose a discontinuity of the SNR curve;

generating, by the control circuitry, a noise model for the HDR imaging system, wherein the noise model between the start transition point and the end transition point is a continuous function based on at least one ratio of exposure times of the plurality of exposures; and

filtering, by the control circuitry, noise from the HDR image using at least the noise model.

9. The method of claim 8 , further comprising:

determining, by the control circuitry, two or more pairs of start transition points and end transition points, wherein each pair encloses a different discontinuity of the SNR curve; and

wherein the noise model between each pair is a continuous function based on at least one ratio of exposure times of the plurality of exposures.

10. The method of claim 8 , further comprising:

generating, by the control circuitry, a combined image signal of the HDR image by summing a plurality of weighted pixel values, wherein each pixel value is associated with a particular exposure of the plurality of exposures.

11. An imaging system comprising:

a pixel array comprising a plurality of pixel cells, wherein each pixel cell is operable to:

generate at least a first image signal and a second image signal, wherein the first image signal is generated in response to the pixel cell being exposed for a first exposure time and the second image signal is generated in response to the pixel cell being exposed for a second exposure time;

an image processor operable to:

generate a noise model for a high dynamic range (“HDR”) image signal of the imaging system, wherein the noise model comprises at least one transition period that is based on the ratio of the first exposure time to the second exposure time; and

a noise filter operable to:

remove at least a portion of noise from the HDR image signal based on at least the noise model.

12. The imaging system of claim 11 , wherein the transition period smoothes a discontinuity of the signal-to-noise ratio of the HDR image signal into a continuous function.

13. The imaging system of claim 11 , wherein the image processor is further operable to generate the high dynamic range image signal from at least the first image signal and the second image signal.

14. The imaging system of claim 11 , wherein the noise filter is a Bayer Filter and is further operable to remove the at least a portion of noise based on local image structure.

15. The imaging system of claim 11 , wherein:

the first and second image signals are generated by a particular pixel cell that senses only green light; and

the noise filter is further operable to remove the at least a portion of noise based on a normalized, weighted sum of image signals generated by pixel cells that sense only green light and that neighbor the particular pixel cell.

16. The imaging system of claim 15 , wherein the pixel cells neighboring the particular pixel cell are located in a 5×5 window surrounding the particular pixel cell.

17. The imaging system of claim 11 , wherein:

the first and second image signal are generated by a particular pixel cell that senses only red light; and

the noise filter is further operable to remove the at least a portion of noise based on:

a plurality of image signals generated by local pixel cells, wherein the local pixel cells sense only red light and neighbor the particular pixel cell; and

a plurality of estimated image signals that would be produced by pixel cells that sense only green light and that are located at the location of each local pixel cell.

18. The imaging system of claim 11 , wherein:

the first and second image signal are generated by a particular pixel cell that senses only blue light; and

the noise filter is further operable to remove the at least a portion of noise based on:

a plurality of image signals generated by local pixel cells, wherein the local pixel cells sense only blue light and neighbor the particular pixel cell; and

a plurality of estimated image signals that would be produced by pixel cells that sense only green light and that are located at the location of each local pixel cell.

Assignments (5)
RELEASE OF SECURITY INTEREST IN PATENTS RECORDED AT REEL 038620, FRAME 0087 Recorded Jun 22, 2023
From: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC; FAIRCHILD SEMICONDUCTOR CORPORATION
Reel/Frame 064070/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE INCORRECT PATENT NUMBER 5859768 AND TO RECITE COLLATERAL AGENT ROLE OF RECEIVING PARTY IN THE SECURITY INTEREST PREVIOUSLY RECORDED ON REEL 038620 FRAME 0087. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY INTEREST. Recorded Aug 25, 2016
From: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
To: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
Reel/Frame 039853/0001 →
SECURITY INTEREST Recorded Apr 15, 2016
From: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
To: DEUTSCHE BANK AG NEW YORK BRANCH
Reel/Frame 038620/0087 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2014
From: APTINA IMAGING CORPORATION
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
Reel/Frame 034037/0711 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 9, 2009
From: LIN, PENG
To: APTINA IMAGING CORPORATION
Reel/Frame 022799/0334 →