IP Library Granted Patent US 9,230,310
Granted Patent B2
US 9,230,310 · App. 14/092,501 · Granted Jan 5, 2016

Imaging systems and methods for location-specific image flare mitigation

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,230,310
App. No.
14/092,501
Granted
Jan 5, 2016
Kind
B2
Abstract

An imaging system may include a camera module with an image sensor having an array of image sensor pixels and one or more lenses that focus light onto the array. The system may include processing circuitry configured to mitigate flare artifacts in image data captured using the array based on at least one image flare map. The image flare map may identify a portion of the captured image data on which to perform image flare mitigation operations. The processing circuitry may perform image flare mitigation operations such as pixel value desaturation on the identified portion of the captured image data without desaturating portions of the image data that do not include flare artifacts. The flare map may be generated using a calibration system that characterizes the location, intensity, and color of all possible image flare artifacts that may be generated by the imaging system during normal imaging operations.

Claims (56)

1. A method for operating an imaging system having a plurality of image sensor pixels and processing circuitry, the method comprising:

with the plurality of image sensor pixels, capturing image data, wherein the imaging system uses adjustable image capture settings when capturing the image data;

with the processing circuitry, identifying an image flare map to be used based on the adjustable image capture settings used when capturing the image data; and

with the processing circuitry, performing image flare mitigation operations on the captured image data based on the identified image flare map.

2. The method defined in claim 1 , wherein performing the image flare mitigation operations comprises:

identifying a portion of the captured image data on which to perform the image flare mitigation operations based on the identified image flare map.

3. The method defined in claim 2 , wherein performing the image flare mitigation operations further comprises:

performing the image flare mitigation operations on the identified portion of the captured image data.

4. The method defined in claim 3 , wherein the image data comprises a plurality of pixel values and wherein performing the image flare mitigation operations on the identified portion of the captured image data comprises:

replacing at least one pixel value in the identified portion of the captured image data with a modified pixel value.

5. The method defined in claim 1 , wherein the identified image flare map comprises a greyscale map of probability values and wherein performing the image flare mitigation operations comprises:

performing the image flare mitigation operations on the captured image data based on the greyscale map.

6. The method defined in claim 5 , wherein the captured image data comprises a plurality of pixel values, wherein the greyscale map comprises a first portion having a first greyscale magnitude and a second portion having a second greyscale magnitude that is less than the first greyscale magnitude, wherein a first set of pixel values in the plurality of pixel values correspond with the first portion of the greyscale map, wherein a second set of pixel values in the plurality of pixel values correspond with the second portion of the greyscale map, and wherein performing the image flare mitigation operations comprises:

applying a first amount of desaturation to the first set of pixel values; and

applying a second amount of desaturation that is less than the first amount of desaturation to the second set of pixel values.

7. The method defined in claim 1 , wherein the captured image data comprises an image frame of pixel values, wherein the image flare map identifies a set of pixel values in the image frame that have a probability of including image flare artifacts that exceeds a threshold probability, and wherein performing the image flare mitigation operations based on the identified image flare map comprises:

performing image flare mitigation operations on the identified set of pixel values.

8. The method defined in claim 1 , further comprising:

with the processing circuitry, storing image flare calibration data; and

with the processing circuitry, retrieving the image flare map from the stored image flare calibration data.

9. A method for calibrating an imaging system using a calibration system, wherein the calibration system comprises a light source and a test host coupled to the imaging system, the method comprising:

with the imaging system, capturing image data from the light source;

with the test host, generating an image flare map using the captured image data; and

with the test host, loading the generated image flare map onto the imaging system.

10. The method defined in claim 9 , further comprising:

placing the light source at a first location relative to the imaging system;

with the imaging system, capturing a first set of image data from the light source at the first location;

placing the light source at a second location relative to the imaging system; and

with the imaging system, capturing a second set of image data from the light source at the second location.

11. The method defined in claim 10 , further comprising:

with the test host, identifying a first set of image flare artifact parameters associated with the first set of image data;

with the test host, identifying a second set of image flare artifact parameters associated with the second set of image data; and

with the test host, combining the identified first and second sets of image flare artifact parameters to generate the image flare map.

12. The method defined in claim 11 , wherein the first set of image flare artifact parameters comprises a first image flare artifact location, wherein the second set of image flare artifact parameters comprises a second image flare artifact location, and wherein combining the first and second sets of image flare artifact parameters comprises:

combining the first and second image flare artifact locations to generate the image flare map.

13. The method defined in claim 10 , wherein capturing the first set of image data from the light source comprises:

capturing an image from the light source at the first position using a set of image capture settings.

14. The method defined in claim 13 , wherein the set of image capture settings comprise at least one image capture setting selected from the group consisting of: an exposure setting, a gain setting, and an aperture setting.

15. The method defined in claim 13 , further comprising:

with the test host, determining whether an image flare artifact is visible in the captured image; and

in response to determining that the image flare artifact is visible in the captured image, determining whether image flare artifact clipping is present in the captured image using the test host.

16. The method defined in claim 15 , further comprising:

with the test host, instructing the imaging system to capture a second image from the light source at the first position using an additional set of image capture settings in response to determining that the image flare artifact clipping is present in the captured image.

17. The method defined in claim 9 , wherein capturing the image data from the light source comprises:

capturing the image data from the light source using a high-dynamic-range imaging mode.

18. A system, comprising:

a central processing unit;

memory;

input-output circuitry; and

an imaging device, wherein the imaging device comprises:

an array of image sensor pixels configured to capture image data;

a lens that focuses an image on the array;

storage and processing circuitry; and

calibration data stored at the storage and processing circuitry, wherein the storage and processing circuitry is configured to identify an image flare mask in the calibration data based on detected image capture settings of the imaging device, and wherein the storage and processing circuitry is configured to perform location-specific image flare mitigation operations on the captured image data based on the identified image flare mask.

19. The system defined in claim 18 , wherein the captured image data comprises a plurality of pixel values at respective pixel value locations, wherein the identified image flare mask identifies selected pixel value locations in the captured image data for performing the location-specific image flare mitigation operations, and wherein the storage and processing circuitry is configured to perform the location-specific image flare mitigation operations at the selected pixel value locations in the captured image data.

20. The system defined in claim 19 , wherein the storage and processing circuitry is configured to perform pixel value desaturation operations only at the selected pixel value locations in the captured image data.

Assignments (5)
RELEASE OF SECURITY INTEREST IN PATENTS RECORDED AT REEL 038620, FRAME 0087 Recorded Jun 22, 2023
From: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC; FAIRCHILD SEMICONDUCTOR CORPORATION
Reel/Frame 064070/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE INCORRECT PATENT NUMBER 5859768 AND TO RECITE COLLATERAL AGENT ROLE OF RECEIVING PARTY IN THE SECURITY INTEREST PREVIOUSLY RECORDED ON REEL 038620 FRAME 0087. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY INTEREST. Recorded Aug 25, 2016
From: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
To: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
Reel/Frame 039853/0001 →
SECURITY INTEREST Recorded Apr 15, 2016
From: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
To: DEUTSCHE BANK AG NEW YORK BRANCH
Reel/Frame 038620/0087 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2014
From: APTINA IMAGING CORPORATION
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
Reel/Frame 034673/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2013
From: BLACK, ROBERT A.; STERN, JONATHAN MICHAEL
To: APTINA IMAGING CORPORATION
Reel/Frame 031688/0414 →