Image sensor read window adjustment for multi-camera array tolerance
View Patent ↗Multiple cameras are arranged in an array at a pitch, roll, and yaw that allow the cameras to have adjacent fields of view such that each camera is pointed inward relative to the array. The read window of an image sensor of each camera in a multi-camera array can be adjusted to minimize the overlap between adjacent fields of view, to maximize the correlation within the overlapping portions of the fields of view, and to correct for manufacturing and assembly tolerances. Images from cameras in a multi-camera array with adjacent fields of view can be manipulated using low-power warping and cropping techniques, and can be taped together to form a final image.
1. A method comprising:
accessing image data captured by an image sensor in each of a plurality of cameras in a camera array, each image sensor comprising an image sensor window and a read window smaller than and located within the image sensor window, the image data from each image sensor representative of light incident upon the read window during capture, wherein at least a portion of a field of view of a first camera is common with a portion of a field of view of a second camera;
determining an amount of correlation between first image data representative of the portion of the field of view of the first camera and second image data representative of the portion of the field of view of the second camera; and
adjusting the location of one or more of the read window of the image sensor of the first camera and the read window of the image sensor of the second camera based on the determined amount of correlation.
2. The method of claim 1 , wherein the camera array is a 2×1 camera array including two cameras.
3. The method of claim 1 , wherein the camera array is a 2×2 camera array including four cameras.
4. The method of claim 1 , wherein the common portion of the fields of view includes a common object.
5. The method of claim 4 , wherein adjusting the location of one or more of the read window of the image sensor of the first camera and the read window of the image sensor of the second camera is based on the common object.
6. The method of claim 4 , wherein adjusting the location of one or more of the read window of the image sensor of the first camera and the read window of the image sensor of the second camera reduces misalignment of the common object in the common portion of the fields of view.
7. The method of claim 4 , wherein the determined amount of correlation comprises a set of correlation coefficients, each representative of one or more portions of the common portion of the fields of view.
8. The method of claim 7 , wherein the set of correlation coefficients are weighted based on the location of the one or more portions of the common portion of the fields of view represented by the set of correlation.
9. The method of claim 8 , wherein weighting the set of correlation coefficients comprises weighting the correlation coefficients representative of locations at or near the center of the common portion of the fields of view more heavily than the correlation coefficients representative of locations more distance from the center of the common portion of the fields of view.
10. The method of claim 8 , wherein weighting the set of correlation coefficients comprises weighting the correlation coefficients representative of locations including the common object more heavily than the correlation coefficients representative of locations not including the common object.
11. The method of claim 1 , wherein a width of the common portion of the fields of view is determined based on a pixel-to-pixel comparison of image data associated with the common portion of the fields of view.
12. The method of claim 1 , wherein the determined amount of correlation is based at least in part on the distance at which a line of sight of each of the plurality of cameras converges.
13. The method of claim 12 , wherein the locations of the one or more read windows are adjusted such that the distance at which the line of sight of each of the plurality of cameras converge is substantially equal to a distance of optimal correlation for the plurality of cameras.
14. A multiple camera system comprising:
a camera array comprising a plurality of cameras each comprising an image sensor, the image sensor comprising an image sensor window and a read window smaller than and located within the image sensor window, the plurality of cameras configured to synchronously capture a set of images such that each camera in the plurality of cameras captures an image, the captured image comprising image data representative of light incident upon the read window of the image sensor of the camera at the time the image is captured, wherein at least a portion of a field of view of a first camera is common with a portion of a field of view of a second camera; and
a calibration controller configured to:
determine an amount of correlation between first image data representative of the portion of the field of view of the first camera and second image data representative of the portion of the field of view of the second camera; and
adjust the location of one or more of the read window of the image sensor of the first camera and the read window of the image sensor of the second camera based on the determined amount of correlation.
15. The multiple camera system of claim 14 , wherein the camera array is a 2×1 camera array including two cameras.
16. The multiple camera system of claim 14 , wherein the camera array is a 2×2 camera array including four cameras.
17. The multiple camera system of claim 14 , wherein the common portion of the fields of view includes a common object.
18. The multiple camera system of claim 17 , wherein adjusting the location of one or more of the read window of the image sensor of the first camera and the read window of the image sensor of the second camera is based on the common object.
19. The multiple camera system of claim 17 , wherein adjusting the location of one or more of the read window of the image sensor of the first camera and the read window of the image sensor of the second camera reduces misalignment of the common object in the common portion of the fields of view.
20. The multiple camera system of claim 17 , wherein the determined amount of correlation comprises a set of correlation coefficients, each representative of one or more portions of the common portion of the fields of view.
21. The multiple camera system of claim 20 , wherein the set of correlation coefficients are weighted based on the location of the one or more portions of the common portion of the fields of view represented by the set of correlation.
22. The multiple camera system of claim 21 , wherein weighting the set of correlation coefficients comprises weighting the correlation coefficients representative of locations at or near the center of the common portion of the fields of view more heavily than the correlation coefficients representative of locations more distance from the center of the common portion of the fields of view.
23. The multiple camera system of claim 21 , wherein weighting the set of correlation coefficients comprises weighting the correlation coefficients representative of locations including the common object more heavily than the correlation coefficients representative of locations not including the common object.
24. The multiple camera system of claim 14 , wherein a width of the common portion of the fields of view is determined based on a pixel-to-pixel comparison of image data associated with the common portion of the fields of view.
25. The multiple camera system of claim 14 , wherein the determined amount of correlation is based at least in part on the distance at which a line of sight of each of the plurality of cameras converges.
26. The multiple camera system of claim 25 , wherein the locations of the one or more read windows are adjusted such that the distance at which the line of sight of each of the plurality of cameras converge is substantially equal to a distance of optimal correlation for the plurality of cameras.