IP Library Granted Patent US 9,224,782
Granted Patent B2
US 9,224,782 · App. 13/971,712 · Granted Dec 29, 2015

Imaging systems with reference pixels for image flare mitigation

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,224,782
App. No.
13/971,712
Granted
Dec 29, 2015
Kind
B2
Abstract

Imaging devices may include processing circuitry, a lens, and an array of image sensor pixels and reference pixels. The array may receive direct image light and stray light from the lens. The image sensor pixels may include clear color filter elements and the reference pixels may include opaque color filter elements. The opaque color filter elements may block direct image light from being captured by the reference pixels. The image sensor pixels may generate pixel values in response to the direct image light and the stray light whereas the reference pixels may generate reference pixel values in response to the stray light. The processing circuitry may mitigate stray light effects such as local flare and veiling glare within the imaging system by adjusting the pixel values based on the reference pixel values. The imaging system may be calibrated in a calibration system for generating stray light calibration data.

Claims (53)

1. A method for operating an imaging system having processing circuitry and an array of image sensor pixels, wherein the array of image sensor pixels includes a reference pixel, the method comprising:

with the array of image sensor pixels, receiving direct image light and stray light;

with the image sensor pixels, generating pixel values in response to the direct image light and the stray light;

with the reference pixel, generating a reference pixel value primarily in response to the stray light; and

with the processing circuitry, adjusting the pixel values generated by the image sensor pixels based on the reference pixel value generated by the reference pixel, wherein adjusting the pixel values comprises:

determining whether the pixel values comprise a texture area; and

in response to determining that the pixel values comprise a texture, performing veiling glare compensation operations on the pixel values.

2. The method defined in claim 1 , wherein the reference pixel comprises a photosensitive region, wherein the image sensor pixels include color filter elements, and wherein generating the reference pixel value primarily in response to the stray light comprises:

generating the reference pixel value in response to receiving the stray light at the photosensitive region of the reference pixel through at least one of the color filter elements.

3. The method defined in claim 2 , wherein the reference pixel further comprises a black color filter element, the method further comprising:

with the black color filter element, blocking the direct image light.

4. The method defined in claim 2 , wherein the reference pixel further comprises a reference pixel color filter element having a metal layer, the method further comprising:

with the metal layer, blocking the direct image light.

5. The method defined in claim 2 , wherein the imaging system comprises a lens, the method further comprising:

with the lens, passing the direct image light onto the array of image sensor pixels within an image circle of the lens, wherein the reference pixel is located on a portion of the array outside of the image circle of the lens.

6. The method defined in claim 2 , wherein the processing circuitry is configured to store stray light calibration data and wherein adjusting the pixel values comprises:

performing local flare compensation operations on the pixel values based on the reference pixel value and the stray light calibration data.

7. The method defined in claim 6 , wherein adjusting the pixel values further comprises:

performing veiling glare compensation operations on the pixel values.

8. The method defined in claim 6 , wherein performing the local flare compensation operations comprises:

identifying pixel value statistics associated with the pixel values; and

computing pixel correction values based on the reference pixel value, the identified pixel value statistics, and the stray light calibration data.

9. The method defined in claim 8 , wherein performing the local flare compensation operations further comprises:

identifying pixel value metadata associated with the pixel values; and

computing the pixel correction values based on the pixel value metadata.

10. The method defined in claim 9 , wherein the pixel value metadata comprises signal-to-noise ratio information and wherein performing the local flare compensation operations further comprises:

computing the pixel correction values based on the signal-to-noise ratio information.

11. The method defined in claim 9 , wherein the pixel value metadata comprises light level information and wherein performing the local flare compensation operations further comprises:

computing the pixel correction values based on the light level information.

12. The method defined in claim 8 , wherein the reference pixel comprises an opaque color filter element and wherein computing the pixel correction values comprises:

computing a perturbed pixel correction value for an image sensor pixel located adjacent to the reference pixel that compensates for light that is blocked by the opaque color filter element.

13. The method defined in claim 2 , wherein the color filter elements of the image sensor pixels comprise clear color filter elements.

14. A method for operating an imaging system having processing circuitry and an array of image sensor pixels, wherein the array of image sensor pixels includes a reference pixel, the method comprising:

with the array of image sensor pixels, receiving direct image light and stray light;

with the image sensor pixels, generating pixel values in response to the direct image light and the stray light;

with the reference pixel, generating a reference pixel value primarily in response to the stray light; and

with the processing circuitry, adjusting the pixel values generated by the image sensor pixels based on the reference pixel value generated by the reference pixel, wherein adjusting the pixel values comprises:

determining whether the pixel values comprise an edge area; and

in response to determining that the pixel values comprise an edge area, performing veiling glare compensation operations on the pixel values.

15. The method defined in claim 14 , further comprising:

with the processing circuitry, replacing the reference pixel value with an interpolated pixel value.

16. The method defined in claim 14 , wherein the reference pixel comprises a photosensitive region, wherein the image sensor pixels include color filter elements, and wherein generating the reference pixel value primarily in response to the stray light comprises:

generating the reference pixel value in response to receiving the stray light at the photosensitive region of the reference pixel through at least one of the color filter elements.

17. A system, comprising:

a central processing unit;

memory;

input-output circuitry; and

an imaging device, wherein the imaging device comprises:

an array of image sensor pixels arranged in rows and columns, wherein the array includes a reference pixel;

a lens that focuses an image onto the array within an image circle of the lens, wherein image sensor pixels in the array that are within the image circle of the lens are configured to receive direct image light and stray light from the lens, wherein the image sensor pixels within the image circle of the lens are configured to generate pixel values in response to the direct image light and the stray light, wherein the reference pixel is located in the array and outside of the image circle of the lens and is configured to generate a reference pixel value in response to the stray light, and wherein the reference pixel is formed without any opaque light shielding elements that shield the reference pixel from the direct light; and

processing circuitry configured to adjust the pixel values based on the reference pixel value.

18. The system defined in claim 17 , wherein the image sensor pixels comprise clear color filter elements.

19. The system defined in claim 18 , wherein the clear color filter elements are arranged in a color filter array, wherein the color filter array comprises repeating groups of red color filter elements, blue color filter elements, and the clear color filter elements.

Assignments (5)
RELEASE OF SECURITY INTEREST IN PATENTS RECORDED AT REEL 038620, FRAME 0087 Recorded Jun 22, 2023
From: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC; FAIRCHILD SEMICONDUCTOR CORPORATION
Reel/Frame 064070/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE INCORRECT PATENT NUMBER 5859768 AND TO RECITE COLLATERAL AGENT ROLE OF RECEIVING PARTY IN THE SECURITY INTEREST PREVIOUSLY RECORDED ON REEL 038620 FRAME 0087. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY INTEREST. Recorded Aug 25, 2016
From: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
To: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
Reel/Frame 039853/0001 →
SECURITY INTEREST Recorded Apr 15, 2016
From: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
To: DEUTSCHE BANK AG NEW YORK BRANCH
Reel/Frame 038620/0087 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2014
From: APTINA IMAGING CORPORATION
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
Reel/Frame 034673/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 20, 2013
From: KEELAN, BRIAN
To: APTINA IMAGING CORPORATION
Reel/Frame 031047/0728 →