Image taping in a multi-camera array
Multiple cameras are arranged in an array at a pitch, roll, and yaw that allow the cameras to have adjacent fields of view such that each camera is pointed inward relative to the array. The read window of an image sensor of each camera in a multi-camera array can be adjusted to minimize the overlap between adjacent fields of view, to maximize the correlation within the overlapping portions of the fields of view, and to correct for manufacturing and assembly tolerances. Images from cameras in a multi-camera array with adjacent fields of view can be manipulated using low-power warping and cropping techniques, and can be taped together to form a final image.
1. A method comprising:
capturing a plurality of images with each camera in a camera array comprising a plurality of cameras, each image comprising at least one portion overlapping with a corresponding portion of a corresponding image;
for each image, performing a first warp operation on the image such that each overlapping portion of the image is substantially aligned with the corresponding overlapping portion of the corresponding image;
for each image, performing a second warp operation on the image to reduce distortion resulting from the first warp operation, wherein a magnitude of the second warp operation on a portion of the image is based on a distance of the portion to the nearest overlapping portion of the image, and wherein the magnitude of the second warp operation on a first portion is greater than the magnitude of the second warp operation on a second portion closer to an overlapping portion than the first portion;
taping each warped image together based on the overlapping portions of the images to form a combined image; and
cropping the combined image to produce a rectangular final image.
2. The method of claim 1 , wherein the plurality of images is captured synchronously.
3. The method of claim 1 , wherein the first warp operation comprises a plurality of local warps performed on portions of each image.
4. The method of claim 1 , wherein a center of the first warp operation performed on an image is based on an overlapping portion of the image.
5. The method of claim 1 , wherein the first warp operation substantially aligns an object of interest within corresponding overlapping portions of a pair of images.
6. The method of claim 1 , wherein the first warp operation performed on an image warps the entire image.
7. The method of claim 1 , wherein the first warp operation performed on an image warps a plurality of portions of the image.
8. The method of claim 1 , wherein the second warp operation performed on an image maps the image to a rectilinear matrix.
9. The method of claim 1 , wherein the second warp operation performed on a pair of images removes distortion in the pair of images while maintaining alignment of an object of interest within overlapping portions of the pair of images.
10. The method of claim 1 , wherein cropping the combined image comprises removing portions of the image that extend past a largest rectangular that can be fully contained within the combined image.
11. A system comprising at least one processor and a non-transitory computer readable medium comprising instructions configured to, when executed by one or more hardware processors, perform steps comprising:
capturing a plurality of images with each camera in a camera array comprising a plurality of cameras, each image comprising at least one portion overlapping with a corresponding portion of a corresponding image;
for each image, performing a first warp operation on the image such that each overlapping portion of the image is substantially aligned with the corresponding overlapping portion of the corresponding image;
for each image, performing a second warp operation on the image to reduce distortion resulting from the first warp operation, wherein a magnitude of the second warp operation on a portion of the image is based on a distance of the portion to the nearest overlapping portion of the image, and wherein the magnitude of the second warp operation on a first portion is greater than the magnitude of the second warp operation on a second portion closer to an overlapping portion than the first portion;
taping each warped image together based on the overlapping portions of the images to form a combined image; and
cropping the combined image to produce a rectangular final image.
12. The system of claim 11 , wherein the plurality of images is captured synchronously.
13. The system of claim 11 , wherein the first warp operation comprises a plurality of local warps performed on portions of each image.
14. The system of claim 11 , wherein a center of the first warp operation performed on an image is based on an overlapping portion of the image.
15. The system of claim 11 , wherein the first warp operation substantially aligns an object of interest within corresponding overlapping portions of a pair of images.
16. The system of claim 11 , wherein the first warp operation performed on an image warps the entire image.
17. The system of claim 11 , wherein the first warp operation performed on an image warps a plurality of portions of the image.
18. The system of claim 11 , wherein the second warp operation performed on an image maps the image to a rectilinear matrix.
19. The system of claim 11 , wherein the second warp operation performed on a pair of images removes distortion in the pair of images while maintaining alignment of an object of interest within overlapping portions of the pair of images.
20. The system of claim 11 , wherein cropping the combined image comprises removing portions of the image that extend past a largest rectangular that can be fully contained within the combined image.