IP Library Granted Patent US 9,769,393
Granted Patent B2
US 9,769,393 · App. 14/856,906 · Granted Sep 19, 2017

Color matching with shade detection

Inventor: Dan G. Allen (Cupertino, CA)
Assignee: Maxim Integrated Products, Inc.
H04N5/2354G06T5/008G06T7/90G06T7/97H04N5/3572G06T2207/10024G06T2207/10152G06T2207/20208
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,769,393
App. No.
14/856,906
Granted
Sep 19, 2017
Kind
B2
Abstract

This disclosure is directed to color matching with shade detection. A method of color matching can include: receiving a camera image of a target, the camera image being collected in the presence of flash illumination; receiving a color sensor spectral measurement of the target, the color sensor spectral measurement being collected in the presence of flash illumination; determining specular and diffuse fractions of a flash intensity profile of the camera image of the target; determining parallax based upon a detected location of a flash centroid within the camera image of the target; converting the parallax to a range measurement for the target; calculating an expected white level for the target based upon the specular and diffuse fractions and the range measurement for the target; and calculating a shade of a detected color based upon the color sensor spectral measurement and the expected white level for the target.

Claims (64)

1. A method of color matching, comprising:

receiving a camera image of a target, the camera image being collected in the presence of flash illumination;

receiving a color sensor spectral measurement of the target, the color sensor spectral measurement being collected in the presence of flash illumination;

determining specular and diffuse fractions of a flash intensity profile of the camera image of the target;

determining parallax based upon a detected location of a flash centroid within the camera image of the target;

converting the parallax to a range measurement for the target;

calculating an expected white level for the target based upon the specular and diffuse fractions and the range measurement for the target; and

calculating a shade of a detected color based upon the color sensor spectral measurement and the expected white level for the target.

2. The method of claim 1 , wherein the shade of the detected color is calculated based upon a ratio of the color sensor spectral measurement and the expected white level for the target.

3. The method of claim 2 , wherein calculating the shade of the detected color includes:

multiplying the ratio of the color sensor spectral measurement and the expected white level for the target by a color correction matrix to obtain tristimulus color values.

4. The method of claim 3 , wherein calculating the shade of the detected color further includes:

normalizing the tristimulus color values.

5. The method of claim 2 , wherein calculating the shade of the detected color further includes:

converting the ratio of the color sensor spectral measurement and the expected white level for the target to a spectral basis by multiplying the ratio of the color sensor spectral measurement and the expected white level for the target by a transformation matrix.

6. The method of claim 1 , wherein the detected color is calculated by dividing a response of the color sensor spectral measurement by a spectral response of the flash illumination.

7. The method of claim 1 , further comprising:

receiving a second camera image of the target, the second camera image being collected in the absence of flash illumination; and

filtering out image effects caused by ambient light from the camera image based upon a difference between the camera image and the second camera image.

8. The method of claim 1 , further comprising:

receiving a second color sensor spectral measurement of the target, the second color sensor spectral measurement being collected in the absence of flash illumination; and

filtering out spectral effects caused by ambient light from the color sensor spectral measurement based upon a difference between the color sensor spectral measurement and the second color sensor spectral measurement.

9. A non-transitory storage medium bearing executable program instructions, including instructions that cause one or more processors to:

receive a camera image of a target, the camera image being collected in the presence of flash illumination;

receive a color sensor spectral measurement of the target, the color sensor spectral measurement being collected in the presence of flash illumination;

determine specular and diffuse fractions of a flash intensity profile of the camera image of the target;

determine parallax based upon a detected location of a flash centroid within the camera image of the target;

convert the parallax to a range measurement for the target;

calculate an expected white level for the target based upon the specular and diffuse fractions and the range measurement for the target; and

calculate a shade of a detected color based upon a ratio of the color sensor spectral measurement and the expected white level for the target.

10. The non-transitory storage medium of claim 9 , wherein the instructions to calculate the shade of the detected color based upon the ratio of the color sensor spectral measurement and the expected white level for the target include instructions to:

multiply the ratio of the color sensor spectral measurement and the expected white level for the target by a color correction matrix to obtain tristimulus color values.

11. The non-transitory storage medium of claim 10 , wherein the instructions to calculate the shade of the detected color based upon the ratio of the color sensor spectral measurement and the expected white level for the target further include instructions to:

normalize the tristimulus color values.

12. The non-transitory storage medium of claim 9 , wherein the instructions to calculate the shade of the detected color based upon the ratio of the color sensor spectral measurement and the expected white level for the target include instructions to:

convert the ratio of the color sensor spectral measurement and the expected white level for the target to a spectral basis by multiplying the ratio of the color sensor spectral measurement and the expected white level for the target by a transformation matrix.

13. The non-transitory storage medium of claim 9 , wherein the program instructions further include instructions to:

divide a response of the color sensor spectral measurement by a spectral response of the flash illumination to calculate the detected color.

14. The non-transitory storage medium of claim 9 , wherein the program instructions further include instructions to:

receive a second camera image of the target, the second camera image being collected in the absence of flash illumination; and

filter out image effects caused by ambient light from the camera image based upon a difference between the camera image and the second camera image.

15. The non-transitory storage medium of claim 9 , wherein the program instructions further include instructions to:

receive a second color sensor spectral measurement of the target, the second color sensor spectral measurement being collected in the absence of flash illumination; and

filter out spectral effects caused by ambient light from the color sensor spectral measurement based upon a difference between the color sensor spectral measurement and the second color sensor spectral measurement.

16. A system for color matching, comprising:

a camera;

a color sensor;

a flash, wherein the camera, the color sensor, and the flash are included in a mobile device, and wherein the camera, the color sensor, and the flash are oriented such that they face the same direction; and

one or more processors configured to:

receive a camera image of a target, the camera image being collected in the presence of flash illumination;

receive a color sensor spectral measurement of the target, the color sensor spectral measurement being collected in the presence of flash illumination; and

report a shade of a color detected by the color sensor, the shade being determined based on data from the camera image of the target and data from the color sensor spectral measurement,

wherein the shade of the detected color is determined by:

determining specular and diffuse fractions of a flash intensity profile of the camera image of the target;

determining parallax based upon a detected location of a flash centroid within the camera image of the target;

converting the parallax to a range measurement for the target;

calculating an expected white level for the target based upon the specular and diffuse fractions and the range measurement for the target; and

calculating the shade of the detected color based upon a ratio of the color sensor spectral measurement and the expected white level for the target.

17. The system of claim 16 , wherein the one or more processors are further configured to:

receive a second camera image of the target, the second camera image being collected in the absence of flash illumination; and

filter out image effects caused by ambient light from the camera image based upon a difference between the camera image and the second camera image.

18. The system of claim 16 , wherein the one or more processors are further configured to:

receive a second color sensor spectral measurement of the target, the second color sensor spectral measurement being collected in the absence of flash illumination; and

filter out spectral effects caused by ambient light from the color sensor spectral measurement based upon a difference between the color sensor spectral measurement and the second color sensor spectral measurement.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 17, 2015
From: ALLEN, DAN G.
To: MAXIM INTEGRATED PRODUCTS, INC.
Reel/Frame 036590/0339 →
Continuity (2)
Provisional Application 62147743 · Apr 15, 2015
Related Publication 20160307336A1 · Oct 20, 2016