Method and apparatus for automatic linear shift parallax correction for multi-array image systems
View Patent ↗A method and apparatus for correcting linear shift parallax error in multi-array image systems. Average pixel cell signal values for pixel cells within a summing window of each column or row of at least two sub-arrays are computed during read-out. The images of the sub-arrays are correlated based on the averages, then shifted based on a result of the correlation function to correct the exhibited parallax error. The summation, correlation, and shifting can be performed by a pixel pipeline processing circuit.
1. A method of correcting parallax error in images acquired from a plurality of pixel cell arrays, said method comprising:
computing a plurality of either column sums or row sums of pixel signal values from each of the plurality of pixel cell arrays;
computing an average pixel signal value corresponding to each of said plurality of sums;
correlating the average pixel signal values of the plurality of pixel cell arrays; and
shifting an image formed from at least one of the plurality of pixel cell arrays based on a result of the correlation.
2. The method of claim 1 , wherein the plurality of pixel cell arrays comprise three pixel cell arrays.
3. The method of claim 2 , wherein a first pixel cell array is sensitive to red light, a second pixel cell array is sensitive to green light, a third pixel cell array is sensitive to blue light.
4. The method of claim 2 , wherein average pixel signal values from a first pixel cell array and a second pixel cell array are correlated, average pixel signal values from the second pixel cell array and a third pixel cell array are correlated, and at least two images formed from the plurality of pixel cell arrays are shifted based on a result of the correlations.
5. The method of claim 1 , wherein the plurality of sums is computed in conjunction with a pixel signal read-out operation of the pixel cell arrays.
6. The method of claim 1 , wherein each of said plurality of sums is a column sum.
7. The method of claim 1 , wherein each of said plurality of sums is a row sum.
8. The method of claim 6 , wherein fewer than all of the pixel signal values in a column are included in each column sum.
9. The method of claim 8 , wherein a first and last group of pixel signal values in each column are excluded from the respective column sum and a middle group of pixel signal values are included in the respective column sum.
10. The method of claim 9 , wherein the middle group of pixel signal values comprises 2 k pixel signal values where k is an integer.
11. The method of claim 10 , wherein each average pixel signal value is computed by shifting the corresponding column sum by k bits.
12. The method of claim 1 , wherein the correlating step comprises determining a maximum of a correlation function.
13. The method of claim 12 , wherein the correlation function is:
corr
(
j
)
=
∑
i
=
j
+
1
N
(
f
i
-
f
_
)
(
g
i
-
j
-
g
_
)
where f and g are average pixel cell values of each column i of pixel cell arrays F and G, respectively, N is a number of columns in each pixel cell array F and G, and f and g are averages of average pixel cell values f i and g i for all columns i of pixel cell arrays F and G, respectively.
14. The method of claim 1 , wherein the shifting step comprises shifting an image along a single axis.
15. A method of correcting linear shift parallax error in an imager having a plurality of linearly-arranged sub-arrays, said method comprising:
reading pixel cell values from a plurality of sub-arrays;
computing sums of pixel values for columns of pixels as said pixel cell values are read-out;
computing average pixel cell values for each of the columns;
correlating images formed from the plurality of sub-arrays based on the average pixel cell values; and
shifting an image formed from at least one of the plurality of sub-arrays based on a result of the correlation.
16. The method of claim 15 , further comprising combining the images formed from the plurality of sub-arrays after at least one of the images has been shifted.
17. The method of claim 15 , wherein the sums and the average pixel cell values are computed by an image processing circuit.
18. The method of claim 17 , wherein the image processing circuit is on the same chip as the plurality of sub-arrays.
19. The method of claim 15 , wherein the correlating and shifting steps are performed by an image processing circuit.
20. The method of claim 17 , wherein the correlating and shifting steps are performed by a central processing unit coupled to the image processing circuit.
21. An imager, comprising:
a plurality of sub-arrays arranged in a linear configuration, each sub-array comprising pixel cells organized in rows and columns;
a read-out circuit for reading out pixel cell values from said plurality of sub-arrays; and
a pixel signal processing circuit configured to compute a sum of the pixel cell values for either columns or rows in each of said plurality of sub-arrays, to compute an average associated with each sum, to correlate images formed from the plurality of sub-arrays based on the averages, and to shift an image formed from at least one of the plurality of sub-arrays in accordance with a result of the correlation.
22. The imager of claim 21 , wherein the pixel signal processing circuit is configured to compute a sum comprising only values of pixel cells within a summing window, wherein the summing window comprises fewer than all of the pixel cells within each column or row.
23. The imager of claim 21 , wherein the pixel signal processing circuit is configured to compute a sum by adding a pixel cell value from the read-out circuit to a partial sum stored in a memory and write the result back to the memory.
24. The imager of claim 21 , wherein the memory is a line buffer memory.
25. The imager of claim 21 , wherein the pixel signal processing circuit is a pixel pipeline processor.
26. The imager of claim 21 , wherein the pixel signal processing circuit is configured to operate at no less than the pixel read-out clock rate.
27. The imager of claim 21 , wherein the result of the correlation is a maximum of a correlation function.
28. The method of claim 27 , wherein the correlation function is:
corr
(
j
)
=
∑
i
=
j
+
1
N
(
f
i
-
f
_
)
(
g
i
-
j
-
g
_
)
where f and g are average pixel cell values of each column i of sub-arrays F and G, respectively, N is a number of columns in each sub-array F and G, and f and g are averages of average pixel cell values f i and g i for all columns i of sub-arrays F and G, respectively.
29. The imager of claim 21 , wherein the plurality of sub-arrays comprises:
a sub-array configured to detect red light;
a sub-array configured to detect green light; and
a sub-array configured to detect blue light.
30. The imager of claim 21 , wherein each of the plurality of sub-arrays is spaced between approximately 2 millimeters and approximately 3 millimeters from an adjacent sub-array.
31. The imager of claim 21 , further comprising an output port configured to output correlated images to a video device.
32. An imager, comprising:
a red sub-array, a green sub-array, and a blue sub-array arranged in a linear configuration, wherein one of the sub-arrays is a center sub-array;
a read-out circuit for reading out pixel cell values from the sub-arrays; and
a pixel processing circuit, comprising:
a summing circuit configured to compute a sum of the pixel cell values for columns in each of the sub-arrays; and
a buffer memory associated with the summing circuit for storing sum values,
wherein the pixel processing circuit is configured to correlate images from a first sub-array and the center sub-array using said sum values in a first correlation, shift an image formed from the first sub-array based on a result of the first correlation, correlate images formed from a second sub-array and the center sub-array using said sum values in a second correlation, shift the image formed from the second sub-array based on a result of the second correlation, and combine the shifted images from said first and second sub-arrays with the image from the center sub-array.
33. The imager of claim 32 , wherein the green sub-array is the center sub-array.
34. The imager of claim 32 , further comprising a lens for focusing an image on said sub-arrays.
35. The imager of claim 34 , further comprising a color filter respectively associated with each sub-array such that each sub-array detects respective wavelengths of light.
36. The imager of claim 32 , wherein the pixel processing circuit is a pixel pipeline processor.
37. A camera system, comprising:
an imager, comprising:
a plurality of sub-arrays arranged in a linear configuration, each sub-array comprising pixel cells organized into rows and columns and each sub-array responsive to a respective band of wavelengths of light;
a read-out circuit for reading out pixel cell values from said plurality of sub-arrays;
a lens for focusing an image on said sub-arrays; and
a pixel signal processing circuit configured to compute a sum of the pixel cell values for each column in each of said sub-arrays, to average the sums of the pixel cell values, to correlate images formed from the plurality of sub-arrays using said averages, to shift an image from at least one of the plurality of sub-arrays based on a result of the correlation, and to combine shifted and unshifted images corresponding to said plurality of sub-arrays.
38. The imager of claim 37 , further comprising an output port configured to output combined images.
39. A method of correcting parallax error in images acquired from a plurality of pixel cell arrays, said method comprising:
computing a plurality of either column sums or row sums of pixel signal values from each of the plurality of pixel cell arrays;
computing an average pixel signal value corresponding to each of said plurality of sums;
correlating the average pixel signal values of the plurality of pixel cell arrays by determining a maximum of a correlation function; and
shifting an image formed from at least one of the plurality of pixel cell arrays based on a result of the correlation,
wherein the correlation function is:
corr
(
j
)
=
∑
i
=
j
+
1
N
(
f
i
-
f
_
)
(
g
i
-
j
-
g
_
)
where f and g are average pixel cell values of each column i of pixel cell arrays F and G, respectively, N is a number of columns in each pixel cell array F and G, and f and g are averages of average pixel cell values f i and g i for all columns i of pixel cell arrays F and G, respectively.
40. An imager, comprising:
a plurality of sub-arrays arranged in a linear configuration, each sub-array comprising pixel cells organized in rows and columns;
a read-out circuit for reading out pixel cell values from said plurality of sub-arrays; and
a pixel signal processing circuit configured to compute a sum of the pixel cell values for either columns or rows in each of said plurality of sub-arrays, to compute an average associated with each sum, to correlate images formed from the plurality of sub-arrays based on the averages, and to shift an image formed from at least one of the plurality of sub-arrays in accordance with a result of the correlation,
wherein the result of the correlation is a maximum of a correlation function, and
wherein the correlation function is:
corr
(
j
)
=
∑
i
=
j
+
1
N
(
f
i
-
f
_
)
(
g
i
-
j
-
g
_
)
where f and g are average pixel cell values of each column i of sub-arrays F and G, respectively, N is a number of columns in each sub-array F and G, and f and g are averages of average pixel cell values f i and g i for all columns i of sub-arrays F and G, respectively.