IP Library › Granted Patent US 12,015,835
Granted Patent B2
US 12,015,835 · App. 17/584,119 · Granted Jun 18, 2024

Multi-sensor imaging color correction

Inventors: Micha Galor Gluskin (San Diego, CA); Jing Wang (Milpitas, CA); Jiafu Luo (Irvine, CA)
Assignee: QUALCOMM Incorporated
H04N23/16G06T7/90H04N9/646H04N9/67G06T2207/10024G06T2207/10048
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,015,835
App. No.
17/584,119
Granted
Jun 18, 2024
Kind
B2
Abstract

Systems and techniques are provided for processing images. For example, a process can include obtaining a first color image including first one or more pixels from a first image sensor and obtaining a second color image including second one or more pixels from a second sensor, the second color image including infrared (IR) information from a second image sensor. The process can include determining a transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels based on a comparison associated with the first one or more pixels and the second one or more pixels. The process can include generating a color corrected image at least in part by transforming the second color image including IR information to a color corrected image based on the determined transformation.

Claims (58)

1. A method for processing images, comprising:

obtaining a first color image including first one or more pixels from a first image sensor, wherein an infrared (IR) cut filter is disposed between a lens of the first image sensor and the first one or more pixels from the first image sensor;

obtaining a second color image including second one or more pixels from a second image sensor, the second color image including IR information from the second image sensor;

determining a transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels based on a comparison associated with the first one or more pixels and the second one or more pixels; and

generating a color corrected image at least in part by transforming the second color image including IR information to a color corrected image based on the determined transformation.

2. The method of claim 1 , wherein a field of view associated with the first one or more pixels at least partially overlaps with a field of view associated with the second one or more pixels.

3. The method of claim 1 , wherein the first image sensor is associated with a first optical axis, and the second image sensor is associated with a second optical axis, different from the first optical axis.

4. The method of claim 1 , wherein the first image sensor has a first resolution, and the second image sensor has a second resolution, different from the first resolution.

5. The method of claim 1 , wherein the first image sensor comprises a single pixel.

6. The method of claim 1 , wherein the first image sensor comprises a plurality of pixels.

7. The method of claim 1 , wherein each of the first one or more pixels includes a corresponding red component, green component, and blue component and each of the second one or more pixels includes a corresponding red component, green component, and blue component, and wherein determining the transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels comprises:

determining a first green statistical measure of green components of the first one or more pixels and a second green statistical measure of green components of the second one or more pixels;

determining a first red statistical measure of red components of the first one or more pixels and a second red statistical measure of red components of the second one or more pixels;

determining a first blue statistical measure of blue components of the first one or more pixels and a second blue statistical measure of blue components of the second one or more pixels;

determining a green color correction based on a numerical correspondence of the first green statistical measure and the second green statistical measure;

determining a red color correction based on a numerical correspondence of the first red statistical measure and the second red statistical measure; and

determining a blue color correction based on a numerical correspondence of the first blue statistical measure and the second blue statistical measure.

8. The method of claim 7 , wherein generating the color corrected image comprises applying the red color correction to the red components of each pixel of the second one or more pixels, applying the green color correction to the green components of each pixel of the second one or more pixels, and applying the blue color correction to the blue components of each pixel of the second one or more pixels.

9. The method of claim 1 , wherein the transformation between colors of the first one or more pixels and colors of the second one or more pixels comprises a color correction matrix.

10. The method of claim 1 , wherein generating the color corrected image comprises applying the determined transformation to the second one or more pixels to produce first color corrected pixels.

11. The method of claim 1 , wherein determining the transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels comprises:

dividing one or more of the first color image and the second color image into a plurality of sections;

determining a first section transformation associated with a first subset of the second one or more pixels associated; and

determining a second section transformation associated with a second subset of the second one or more pixels, wherein the second subset of the second one or more pixels differs from the first subset by at least one pixel.

12. The method of claim 11 , further comprising determining a global transformation for generating the color corrected image based on the first section transformation and the second section transformation.

13. The method of claim 11 , further comprising generating the color corrected image at least in part by transforming the first subset of the second one or more pixels by the first section transformation and transforming the second subset of the second one or more pixels by the second section transformation.

14. The method of claim 1 , wherein the first one or more pixels do not include IR information.

15. An apparatus for processing images, comprising:

a memory; and

one or more processors coupled to the memory and configured to:

obtain a first color image including first one or more pixels from a first image sensor, wherein an infrared (IR) cut filter is disposed between a lens of the first image sensor and the first one or more pixels from the first image sensor;

obtain a second color image including second one or more pixels from a second image sensor, the second color image including IR information from the second image sensor;

determine a transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels based on a comparison associated with the first one or more pixels and the second one or more pixels; and

generate a color corrected image at least in part by transform the second color image including IR information to a color corrected image based on the determined transformation.

16. The apparatus of claim 15 , wherein a field of view associated with the first one or more pixels at least partially overlaps with a field of view associated with the second one or more pixels.

17. The apparatus of claim 15 , wherein the first image sensor is associated with a first optical axis, and the second image sensor is associated with a second optical axis, different from the first optical axis.

18. The apparatus of claim 15 , wherein the first image sensor has a first resolution, and the second image sensor has a second resolution, different from the first resolution.

19. The apparatus of claim 15 , wherein the first image sensor comprises a single pixel.

20. The apparatus of claim 15 , wherein the first image sensor comprises a plurality of pixels.

21. The apparatus of claim 15 , wherein:

each of the first one or more pixels includes a corresponding red component, green component, and blue component and each of the second one or more pixels includes a corresponding red component, green component, and blue component; and

to determine the transformation between colors associated with the first one or more pixels and colors associated with the second one or more pixels, the one or more processors are configured to:

determine a first green statistical measure of green components of the first one or more pixels and a second green statistical measure of green components of the second one or more pixels;

determine a first red statistical measure of red components of the first one or more pixels and a second red statistical measure of red components of the second one or more pixels;

determine a first blue statistical measure of blue components of the first one or more pixels and a second blue statistical measure of blue components of the second one or more pixels;

determine a green color correction based on a numerical correspondence of the first green statistical measure and the second green statistical measure;

determine a red color correction based on a numerical correspondence of the first red statistical measure and the second red statistical measure; and

determine a blue color correction based on a numerical correspondence of the first blue statistical measure and the second blue statistical measure.

22. The apparatus of claim 21 , wherein, to generate the color corrected image, the one or more processors are configured to apply the red color correction to the red components of each pixel of the second one or more pixels, apply the green color correction to the green components of each pixel of the second one or more pixels, and apply the blue color correction to the blue components of each pixel of the second one or more pixels.

23. The apparatus of claim 15 , wherein the transformation between colors of the first one or more pixels and colors of the second one or more pixels comprises a color correction matrix.

24. The apparatus of claim 15 , wherein, to generate the color corrected image, the one or more processors are configured to apply the determined transformation to the second one or more pixels to produce first color corrected pixels.

25. The apparatus of claim 15 , wherein the one or more processors are configured to:

divide one or more of the first color image and the second color image into a plurality of sections;

determine a first section transformation associated with a first subset of the second one or more pixels associated; and

determine a second section transformation associated with a second subset of the second one or more pixels, wherein the second subset of the second one or more pixels differs from the first subset by at least one pixel.

26. The apparatus of claim 25 , wherein the one or more processors are configured to: determine a global transformation for generate the color corrected image based on the first section transformation and the second section transformation.

27. The apparatus of claim 25 , wherein the one or more processors are configured to: generate the color corrected image at least in part by transform the first subset of the second one or more pixels by the first section transformation and transform the second subset of the second one or more pixels by the second section transformation.

28. The apparatus of claim 15 , wherein the first one or more pixels do not include IR information.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2022
From: GALOR GLUSKIN, MICHA; WANG, JING; LUO, JIAFU
To: QUALCOMM INCORPORATED
Reel/Frame 058932/0499 →
Continuity (1)
Related Publication 20230239553A1 · Jul 27, 2023
Cited By (1)
US 12,244,936