Image sensor read window adjustment for multi-camera array tolerance
View Patent ↗Multiple cameras are arranged in an array at a pitch, roll, and yaw that allow the cameras to have adjacent fields of view such that each camera is pointed inward relative to the array. The read window of an image sensor of each camera in a multi-camera array can be adjusted to minimize the overlap between adjacent fields of view, to maximize the correlation within the overlapping portions of the fields of view, and to correct for manufacturing and assembly tolerances. Images from cameras in a multi-camera array with adjacent fields of view can be manipulated using low-power warping and cropping techniques, and can be taped together to form a final image.
1. A method comprising:
accessing image data captured by an image sensor in each of a plurality of cameras in a camera array, each image sensor comprising an image sensor window and a read window smaller than and located within the image sensor window, the image data from each image sensor representative of light incident upon the read window during capture, a first camera in the camera array including a first portion of a first field of view that overlaps with a second portion of a second field of view of a second camera in the camera array;
identifying a portion of the accessed image data representative of the first portion of the first field of view and a portion of the accessed image data representative of the second portion of the second field of view;
determining a set of correlation coefficients for the identified portions of image data representative of an amount of correlation within the image data portions; and
responsive to the set of correlation coefficients representing a below-threshold level of correlation, adjusting the location of one or more of the read window of the image sensor of the first camera and the read window of the image sensor of the second camera.
2. The method of claim 1 , wherein the camera array is a 2×1 camera array including two cameras.
3. The method of claim 1 , wherein the camera array is a 2×2 camera array including four cameras.
4. The method of claim 1 , wherein the overlapping first portion of the first field of view and second portion of the second field of view include a common object.
5. The method of claim 4 , wherein adjusting the location of one or more of the read window of the first camera and the read window of the image sensor of the second camera is based on the common object in both the first and second portion.
6. The method of claim 4 , wherein adjusting the location of one or more of the read window of the image sensor of the first camera and the read window of the image sensor of the second camera reduces misalignment of the common object in both the first and second portion.
7. The method of claim 1 , wherein the overlapping first portion of the first field of view and second portion of the second field of view are automatically identified.
8. The method of claim 7 , wherein the overlapping first portion of the first field of view and second portion of the second field of view is identified based on a pixel-to-pixel comparison of image sensor readouts of the first portion and the second portion.
9. The method of claim 1 , further comprising:
identifying a distance of optimal correlation for the plurality of cameras in the camera array;
wherein adjusting the location of one or more of the read window of the image sensor of the first camera and the read window of the image sensor of the second camera is based additionally on the identified distance of optimal correlation.
10. The method of claim 1 , further comprising:
after adjusting the location of one or more of the read window of the image sensor of the first camera and the read window of the image sensor of the second camera, accessing subsequent image data captured by the first camera and the second camera;
identifying a portion of the accessed subsequent image data representative of the first portion of the first field of view and a portion of the subsequent accessed image data representative of the second portion of the second field of view;
determining a subsequent set of correlation coefficients for the identified portions of subsequent accessed image data representative of an amount of correlation within the identified portions of subsequent accessed image data portions;
responsive to the subsequent set of correlation coefficients representing a below-threshold level of correlation, further adjusting the location of one or more of the read window of the image sensor of the first camera and the read window of the image sensor of the second camera; and
responsive to the subsequent set of correlation coefficients representing an above-threshold level of correlation, storing the location of the read window of the image sensor of the first camera and the location of the read window of the image sensor of the second camera for subsequent use in capturing images by the camera array.
11. A method comprising:
accessing image data captured by an image sensor in each of a plurality of cameras in a camera array, each image sensor comprising an image sensor window and a read window smaller than and located within the image sensor window, the image data from each image sensor representative of light incident upon the read window during capture, each camera in the camera array including a first portion of a first field of view that overlaps with a second portion of a second field of view of at least one other associated camera in the camera array, the associated cameras horizontally or vertically adjacent to each other;
identifying a portion of the accessed image data representative of the first portion of the first field of view and a portion of the accessed image data representative of the second portion of the second field of view;
determining an amount of correlation within the image data portions; and
adjusting the location of at least one of the read windows of the image sensors of the camera and the another camera in the camera array to reduce entropy within the image data portions.
12. The method of claim 11 , wherein the camera array is a 2×1 camera array including two cameras.
13. The method of claim 11 , wherein the camera array is a 2×2 camera array including four cameras.
14. The method of claim 11 , wherein the overlapping first portion of the first field of view and second portion of the second field of view include a common object.
15. The method of claim 14 , wherein adjusting the location of one or more of the read window of the camera and the read window of the image sensor of the another camera is based on the common object in both the first and second portion.
16. The method of claim 14 , wherein adjusting the location of one or more of the read window of the image sensor of the camera and the read window of the image sensor of the another camera reduces misalignment of the common object in both the first and second portion.
17. The method of claim 11 , wherein the overlapping first portion of the first field of view and second portion of the second field of view are automatically identified.
18. The method of claim 17 , wherein the overlapping first portion of the first field of view and second portion of the second field of view is identified based on a pixel-to-pixel comparison of image sensor readouts of the first portion and the second portion.
19. The method of claim 11 , further comprising:
identifying a distance of optimal correlation for the plurality of cameras in the camera array; and
adjusting the location of one or more of the read window of the camera and the read window of the image sensor of the another camera based on the identified distance of optimal correlation.
20. The method of claim 11 , further comprising storing the location of the read windows of the image sensors of the plurality of cameras in the camera array for future use in capturing images by the camera array.
21. A system comprising at least one processor and a non-transitory computer readable medium comprising instructions for adjusting read windows in image sensors of a plurality of cameras in a camera array, the instructions executable by the at least one processor, the instructions comprising instructions for:
accessing image data captured by an image sensor in each of a plurality of cameras in a camera array, each image sensor comprising an image sensor window and a read window smaller than and located within the image sensor window, the image data from each image sensor representative of light incident upon the read window during capture, each camera in the camera array including a first portion of a first field of view that overlaps with a second portion of a second field of view of at least one other associated camera in the camera array, the associated cameras horizontally or vertically adjacent to each other;
identifying a portion of the accessed image data representative of the first portion of the first field of view and a portion of the accessed image data representative of the second portion of the second field of view;
determining an amount of correlation within the image data portions; and
adjusting the location of at least one of the read windows of the image sensors of the camera and the another camera in the camera array to reduce entropy within the image data portions.
22. The system of claim 21 , wherein the camera array is a 2×1 camera array including two cameras.
23. The system of claim 21 , wherein the camera array is a 2×2 camera array including four cameras.
24. The system of claim 21 , wherein the overlapping first portion of the first field of view and second portion of the second field of view include a common object.
25. The system of claim 24 , wherein adjusting the location of one or more of the read window of the camera and the read window of the image sensor of the another camera is based on the common object in both the first and second portion.
26. The system of claim 24 , wherein adjusting the location of one or more of the read window of the image sensor of the camera and the read window of the image sensor of the another camera reduces misalignment of the common object in both the first and second portion.
27. The system of claim 21 , wherein the overlapping first portion of the first field of view and second portion of the second field of view are automatically identified.
28. The system of claim 27 , wherein the overlapping first portion of the first field of view and second portion of the second field of view is identified based on a pixel-to-pixel comparison of image sensor readouts of the first portion and the second portion.
29. The system of claim 21 , further comprising:
identifying a distance of optimal correlation for the plurality of cameras in the camera array; and
adjusting the location of one or more of the read window of the camera and the read window of the image sensor of the another camera based on the identified distance of optimal correlation.
30. The system of claim 21 , further comprising storing the location of the read windows of the image sensors of the plurality of cameras in the camera array for future use in capturing images by the camera array.