Method of color filter design and color reproduction under the effect of pixel crosstalk in CMOS image sensors
View Patent ↗The present invention is directed at method of designing a Color Filter Arrays (CFA) for CMOS image sensors under the effects of crosstalk for optimal color reproduction. Instead of a focus on lowering crosstalk, a novel method of designing color filter spectral responses to compensate for the effect of crosstalk at the color imaging system level is proposed. As part of this method, a color reproduction model for CMOS and CCD image sensor under the effect of crosstalk is also proposed.
1. A method of designing a color filter array, having spectral response curves for all color channels, for digital imaging sensors under the effect of crosstalk comprising:
determining a crosstalk characteristic function using cumulative effects of up to all pixels on a image sensor;
modeling color reproduction based on said crosstalk characteristic function and said spectral response curves;
optimizing said spectral response curves of said color filter array wherein the shape of the spectral response curves for as many as all color channels extant on the image sensor are selected to optimize the accuracy of color reproduction; and,
inputting the optimized spectral response curves of said color filter array back to the determining crosstalk characteristic function and generating a color reproduction model steps until a predetermined metric threshold is reached or exceeded by some predetermined amount.
2. The method of claim 1 wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor is calculated in said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on decreasing the difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor calculated in said modeling color reproduction step.
3. The method of claim 1 wherein said determining crosstalk characteristic functions step is achieved by direct measurement of the spectral response of individual pixels as a function of wavelength using a monochromator to step through a range of wavelengths to illuminate said pixels one wavelength band at a time.
4. The method of claim 3 wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor is calculated in said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on decreasing the difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor calculated in said modeling color reproduction step.
5. The method of claim 3 wherein light from said monochromator is focused so that only a single pixel is illuminated at a time.
6. The method of claim 5 wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor is calculated in said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on decreasing the difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor calculated in said modeling color reproduction step.
7. The method of claim 3 wherein a metal mask is applied to the digital imaging sensor so that only one pixel is exposed at a time.
8. The method of claim 7 wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor is calculated in said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on decreasing the difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor calculated in said modeling color reproduction step.
9. The method of claim 1 wherein said determining crosstalk characteristic functions step is achieved by simulation.
10. The method of claim 9 wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor is calculated in said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on decreasing the difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor calculated in said modeling color reproduction step.
11. The method of claim 9 wherein said simulation comprises;
constructing a process flow including all major mask steps and thermal cycles using a process simulator;
calibrating said process flow against Secondary Ion Mass Spectrometry data; and,
entering said calibrated process flow into a device simulator;
constructing the crosstalk characteristic function using said device simulator.
12. The method of claim 11 wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor is calculated in said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on decreasing the difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor calculated in said modeling color reproduction step.
13. The method of claim 1 wherein said crosstalk characteristic function is approximated as linear function CT(λ)=a+b·λ wherein
a
=
λ
n
·
CT
min
-
λ
1
·
CT
max
λ
max
-
λ
1
and
b
=
CT
max
-
CT
min
λ
n
-
λ
1
where the λ l and λ n represent the extremes of the range of wavelengths, CT min and CT max are the minimum and maximum values of crosstalk corresponding to λ l and λ n , respectively.
14. The method of claim 1 wherein said crosstalk characteristic function is set of piece wise linear functions wherein each linear function is CT b (λ)=a b +b b ·λ wherein
a
=
λ
bn
·
CT
min
b
-
λ
b
1
·
CT
max
b
λ
max
b
-
λ
1
b
and
b
b
=
CT
max
b
-
CT
min
b
λ
nb
-
λ
1
b
where the λ lb and λ nb represent the extremes of the range of wavelengths of the b th band of the overall wavelength range of the crosstalk characteristic function, CT minb and CT mabx are the minimum and maximum values of crosstalk correpsonding to λ lb and λ nb , respectively.
15. The method of claim 14 wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor is calculated in said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on decreasing the difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor calculated in said modeling color reproduction step.
16. The method of claim 1 wherein said modeling color reproduction step is based on the equations R ct (λ)=R rr (λ)+G gr (λ)+B br (λ), G ct (λ)=R rg (λ)+G gg (λ)+B bg (λ), and B ct (λ)=R rb (λ)+G gb (λ)+B bb (λ) where R ct (λ), G ct (λ), B ct (λ) are the respective composite spectral sensitivities for red, green and blue pixels in a Bayer color filter array (CFA), R rr (λ), G gg (λ), B bb (λ) are the primary spectral sensitivities components of red, green and bluish pixels respectively, R rg (λ) is the spectral sensitivity contribution of red pixels to green pixels due to crosstalk, R rb (λ) is the spectral sensitivity contribution of red pixels to blue pixels due to crosstalk, G gr (λ) is the spectral sensitivity contribution of green pixels to red pixels due to crosstalk, G gb (λ) is the spectral sensitivity contribution of green pixels to blue pixels due to crosstalk, B br (λ) is the spectral sensitivity contribution of blue pixels to red pixels due to crosstalk and B bg (λ) is the spectral sensitivity contribution of blue pixels to green pixels due to crosstalk.
17. The method of claim 16 wherein a value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor is calculated in said modeling color reproduction step, wherein said optimizing said spectral response curves of said color filter array step is based on decreasing the difference between a predetermined threshold value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor and said value of q-factor, μ-factor, Q st and Q sf , Figure of Merit, Unified Measure of Goodness or other quality metric of the goodness of spectral sensitivities of an image sensor calculated in said modeling color reproduction step.
18. The method of claim 1 wherein said modeling color reproduction step is based on said crosstalk characteristic function and said spectral response curves are only those available to a particular color filter array manufacturing process.