IP Library Granted Patent US 10,798,311
Granted Patent B2
US 10,798,311 · App. 16/267,897 · Granted Oct 6, 2020

Methods and apparatus for reducing spatial flicker artifacts

Inventors: Manjunath Somayaji (Santa Clara, CA); Rohit Ranade (Milpitas, CA)
Assignee: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
H04N5/2357G09G5/026G09G5/377H04N5/2351H04N5/2355H04N9/045G09G2320/02G09G2320/0247
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,798,311
App. No.
16/267,897
Granted
Oct 6, 2020
Kind
B2
Abstract

Various embodiments of the present technology may comprise a method and apparatus for reducing spatial flicker artifacts in high dynamic range images produced from multiple image captures with differing integration times. The method and apparatus may comprise, in the image captures, measuring pixel intensity selecting pixels with a predetermined intensity thresholds, calculating a ratio based on a channel selection, normalizing the calculated ratio to an ideal ratio, and applying the normalized ratio to corresponding image pixels in a second image capture to produce a flicker-free image.

Claims (68)

1. An imaging device comprising:

an array of pixels arranged in rows and columns, wherein the array of pixels generates:

first image pixel data corresponding to pixels having a first exposure time, and

second image pixel data corresponding to pixels having a second exposure time,

wherein the first exposure time is greater than the second exposure time; and

wherein the pixels used to generate the first and second image pixel data have the same spatial locations within the array;

a color filter overlaying the array pixels and comprising a plurality of color filters; and

an image signal processor coupled to the array of pixels, wherein the image signal processor:

receives the first image pixel data and second image pixel data;

identifies valid and invalid image pixel data based on pixel data values;

generate a ratio utilizing the valid image pixel data;

wherein:

the ratio comprises a first individual pixel value divided by a second individual pixel value;

the first individual pixel value corresponds to a first color from the plurality of color filters;

the second individual pixel value corresponds to a second color from the plurality of color filters that is different from the first color; and

the first and second individual pixel values have a same exposure time;

normalizes the computed ratio using an ideal ratio, wherein the ideal ratio is the first exposure time divided by the second exposure time; and

modifies the second image pixel data using the normalized computed ratio.

2. The imaging device of claim 1 , wherein the invalid pixel data comprises:

first image pixel data with a value above a first predetermined threshold, and

second image pixel data with a value below a second predetermined threshold.

3. The imaging device of claim 1 , wherein the valid pixel data comprises first image pixel data with a value below a first predetermined threshold.

4. The imaging device of claim 3 , wherein the valid pixel data further comprises second image pixel data with a value above a second predetermined threshold.

5. The imaging device of claim 1 , wherein the image signal processor further identifies the valid image pixel data within a neighborhood of each identified invalid image pixel data.

6. The imaging device of claim 5 , wherein the image signal processor computes a representative value for the identified valid image pixel data within the neighborhood of the identified invalid image pixel data and replacing the identified invalid image pixel data with the representative value.

7. The imaging device of claim 1 , wherein the first and second individual pixel values are generated by pixels that share at least one of: a same row and a same column.

8. The imaging device of claim 1 , wherein the first and second individual pixel values are generated by pixels that are directly adjacent to each other.

9. A method for reducing spatial artifacts in multi-exposure images comprising:

generating, with an image sensor comprising an array of color pixels arranged in rows and columns, first image pixel data corresponding to pixels having a first exposure time, and second image pixel data corresponding to pixels having a second exposure time, wherein the first exposure time is greater than the second exposure time; wherein the pixels used to generate the first and second image pixel data have the same spatial locations within the array; wherein the first and second image pixel data comprise a value representing light intensity;

identifying whether the value is one of a valid or an invalid image pixel data;

generating a ratio utilizing the valid image pixel data;

wherein:

the ratio comprises a first individual pixel value divided by a second individual pixel value;

the first individual pixel value corresponds to a first color;

the second individual pixel value corresponds to a second color that is different from the first color; and

the first and second individual pixel values have a same exposure time;

normalizing the computed ratio utilizing an ideal ratio, wherein the ideal ratio is the first exposure time divided by the second exposure time; and

modifying the second image pixel data using the normalized ratio.

10. The method of claim 9 , wherein the value is invalid if the value is greater than a first predetermined threshold.

11. The method of claim 10 , wherein the value is invalid if the value is less than a second predetermined threshold.

12. The method of claim 9 , wherein the value is valid if the value is between a first predetermined threshold and a second predetermined threshold.

13. The method of claim 9 , wherein the image signal processor further identifies the valid image pixel data within a neighborhood of each identified invalid image pixel data.

14. The method of claim 13 , wherein the image signal processor computes a representative value for the valid image pixel within a neighborhood of the invalid pixel and replacing the invalid pixel value with the average pixel value.

15. The method of claim 9 , further comprising a color filter array disposed over the array of pixels, wherein the color filter array comprises the first color and the second color.

16. The method of claim 9 , wherein the first and second individual pixel values are generated by pixels that share at least one of: a same row and a same column.

17. The method of claim 9 , wherein the first and second individual pixel values are generated by pixels that are directly adjacent to each other.

18. A system for producing flicker-free images of a scene illuminated by a light using an alternating current source, comprising:

an image sensor, comprising:

a color filter array comprising a plurality of color filters situated on an array of pixels arranged in rows and columns, wherein the image sensor obtains:

first image pixel data corresponding to pixels having a first exposure time, and

second image pixel data corresponding to pixels having a second exposure time;

wherein the first exposure time is greater than the second exposure time;

wherein the pixels used to generate the first and second image pixel data have the same spatial locations within the array;

a storage unit coupled to the image sensor, wherein the storage unit stores the first and second image pixel data;

an image signal processor coupled to the storage unit, wherein the image signal processor:

identifies valid and invalid image pixel data based on a pixel data value;

generates a color ratio utilizing the valid image pixel data;

wherein:

the ratio comprises a first individual pixel value divided by a second individual pixel value;

the first individual pixel value corresponds to a first color from the plurality of color filters;

the second individual pixel value corresponds to a second color from the plurality of color filters that is different from the first color; and

the first and second individual pixel values have a same exposure time;

normalizes the color ratio utilizing an ideal ratio, wherein the ideal ratio is the first exposure time divided by the second exposure time;

modifies the second image pixel data using the normalized color ratio to form corrected image pixel data; and

combines the corrected image pixel data with the first image pixel data to form a multi-pixel data image; and

an I/O device for displaying the multi-pixel data image.

19. The system according to claim 18 , wherein the first and second individual pixel values are generated by pixels that share at least one of: a same row and a same column.

20. The system according to claim 18 , wherein the first and second individual pixel values are generated by pixels that are directly adjacent to each other.

Assignments (3)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL 049473, FRAME 0932 Recorded Aug 17, 2023
From: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC; FAIRCHILD SEMICONDUCTOR CORPORATION
Reel/Frame 064618/0721 →
SECURITY INTEREST Recorded Jun 14, 2019
From: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC; FAIRCHILD SEMICONDUCTOR CORPORATION
To: DEUTSCHE BANK AG NEW YORK BRANCH, AS COLLATERAL AGENT
Reel/Frame 049473/0932 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 5, 2019
From: SOMAYAJI, MANJUNATH; RANADE, ROHIT
To: SEMICONDUCTOR COMPONENTS INDUSTRIES, LLC
Reel/Frame 048241/0332 →