IP Library Granted Patent US 9,497,427
Granted Patent B2
US 9,497,427 · App. 13/942,201 · Granted Nov 15, 2016

Method and apparatus for image flare mitigation

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,497,427
App. No.
13/942,201
Granted
Nov 15, 2016
Kind
B2
Abstract

An imaging system may include a camera module with an image sensor having an array of image sensor pixels and one or more lenses that focus light onto the array of image sensor pixels. The array of image sensor pixels may include a corresponding array of color filter elements. The system may include circuitry configured to detect and mitigate flare artifacts in image data captured using the image sensor. Detecting the flare artifacts may include detecting color mismatches in the captured image data and performing edge-detection operations to determine whether the color mismatches are in an edge region of the image data. If the color mismatches are detected in an edge region, no flare-mitigation operations may be performed. If the color mismatches are in a flat region of the captured image, flare-mitigation operations may be performed.

Claims (35)

1. An imaging system, comprising:

an image sensor comprising a plurality of image pixels, wherein the plurality of image pixels are configured to capture image data;

at least one lens that focuses image light onto the image sensor; and

processing circuitry configured to detect and mitigate flare artifacts in a frame of the captured image data by detecting color mismatches in portions of the frame of the captured image data captured by at least first and second image pixels of the plurality of image pixels, wherein the processing circuitry detects color mismatches in the portions of the frame of the captured image data that is output from the first and second image pixels by comparing a ratio between a first portion of the frame of the captured image data captured by the first image pixel and a second portion of the frame of the captured image data captured by the second image pixel to a predetermined threshold.

2. The imaging system defined in claim 1 wherein the processing circuitry is further configured to detect the flare artifacts by performing edge-detection operations on the frame of the captured image data.

3. The imaging system defined in claim 2 wherein image sensor comprises an array of image pixels and a corresponding array of color filter elements.

4. The imaging system defined in claim 3 wherein the corresponding array of color filter elements comprises repeating groups of red color filter elements, blue color filter elements, and green color filter elements.

5. The imaging system defined in claim 3 wherein the corresponding array of color filter elements comprises repeating groups of red color filter elements, blue color filter elements, and clear color filter elements.

6. The imaging system defined in claim 2 wherein image sensor comprises a plurality of arrays of image pixels and wherein each array of image pixels has a corresponding array of color filter elements having a common color.

7. The imaging system defined in claim 1 , wherein the first image pixel is adjacent to the second image pixel.

8. A method of generating images, comprising:

with an image sensor comprising a plurality of image pixels, capturing an image having image data;

with processing circuitry, detecting color mismatches in portions of the captured image data that is output from at least first and second clear image pixels of the plurality of image pixels; and

with the processing circuitry, detecting flare artifacts in at least some of the portions of the captured image data based on the detected color mismatches by comparing a difference between an image signal from the first clear image pixel and an additional image signal from the second clear image pixel to a predetermined threshold.

9. The method defined in claim 8 , further comprising:

with the processing circuitry, performing edge-detection operations on the captured image data.

10. The method defined in claim 9 wherein performing the edge-detection operations on the captured image data comprises performing the edge-detection operations on the captured image data prior to detecting the color mismatches in the portions of the captured image data.

11. The method defined in claim 9 wherein performing the edge-detection operations on the captured image data comprises performing the edge-detection operations on the captured image data after detecting the color mismatches in the portions of the captured image data.

12. The method defined in claim 9 , further comprising:

with the processing circuitry, performing flare-mitigation operations on the at least some of the portions of the captured image data.

13. The method defined in claim 12 wherein performing the flare-mitigation operations on the at least some of the portions of the captured image data comprises generating flare-indicating metadata.

14. The method defined in claim 12 wherein performing the flare-mitigation operations on the at least some of the portions of the captured image data comprises desaturating the at least some of the portions of the captured image data.

15. The method defined in claim 12 wherein performing the flare-mitigation operations on the at least some of the portions of the captured image data comprises modifying the at least some of the portions of the captured image data using a flare-specific color correction matrix.

16. The method defined in claim 8 , further comprising:

computing a ratio of the image signal and the additional image signal; and

comparing the computed ratio to an additional threshold.

17. A system, comprising:

a central processing unit;

memory;

input-output circuitry; and

an imaging device, wherein the imaging device comprises:

an image sensor having an array of image sensor pixels arranged in repeating unit cells each having a red pixel, a blue pixel, and two clear pixels;

at least one lens that focuses image light onto the image sensor; and

processing circuitry configured to detect flare artifacts in an image having image data captured using the image sensor by detecting color mismatches in a portion of the captured image data of the image captured by at least first and second image sensor pixels, wherein the processing circuitry is further configured to detect edges of objects that are being imaged in the portion of the captured image data, wherein the processing circuitry is further configured to perform flare mitigation operations when no object edges are detected in the portion of the captured image data, and wherein the processing circuitry is further configured to omit flare-mitigation operations on the captured image data when object edges are detected in the portion of the captured image data.

18. The system defined in claim 17 wherein the processing circuitry is further configured to perform flare-mitigation operations on the detected flare artifacts in response to determining that the portion of the captured image data is flat.

Assignments (5)
RELEASE OF SECURITY INTEREST IN PATENTS RECORDED AT REEL 038620, FRAME 0087 Recorded Jun 22, 2023
From: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC; FAIRCHILD SEMICONDUCTOR CORPORATION
Reel/Frame 064070/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE INCORRECT PATENT NUMBER 5859768 AND TO RECITE COLLATERAL AGENT ROLE OF RECEIVING PARTY IN THE SECURITY INTEREST PREVIOUSLY RECORDED ON REEL 038620 FRAME 0087. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY INTEREST. Recorded Aug 25, 2016
From: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
To: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
Reel/Frame 039853/0001 →
SECURITY INTEREST Recorded Apr 15, 2016
From: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
To: DEUTSCHE BANK AG NEW YORK BRANCH
Reel/Frame 038620/0087 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2014
From: APTINA IMAGING CORPORATION
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
Reel/Frame 034673/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 15, 2013
From: STERN, JONATHAN MICHAEL; BLACK, ROBERT A.
To: APTINA IMAGING CORPORATION
Reel/Frame 030799/0112 →