Image taping in a multi-camera array
View Patent ↗Multiple cameras are arranged in an array at a pitch, roll, and yaw that allow the cameras to have adjacent fields of view such that each camera is pointed inward relative to the array. The read window of an image sensor of each camera in a multi-camera array can be adjusted to minimize the overlap between adjacent fields of view, to maximize the correlation within the overlapping portions of the fields of view, and to correct for manufacturing and assembly tolerances. Images from cameras in a multi-camera array with adjacent fields of view can be manipulated using low-power warping and cropping techniques, and can be taped together to form a final image.
1. A method comprising:
capturing a plurality of images with each camera in a camera array comprising a plurality of cameras, each image comprising at least one portion overlapping with a corresponding portion of a corresponding image;
aligning overlapping portions of corresponding images to produce a set of aligned images;
for each aligned image, performing a warp operation on the aligned image to produce a warped image, wherein a magnitude of the warp operation on a portion of the aligned image increases with an increase in distance from the portion of the aligned image to the nearest overlapping portion of the aligned image;
taping each warped image together based on the overlapping portions of the warped images to form a combined image; and
cropping the combined image to produce a rectangular final image.
2. The method of claim 1 , wherein the plurality of images is captured synchronously.
3. The method of claim 1 , wherein the warp operation comprises a plurality of local warps performed on portions of each aligned image.
4. The method of claim 1 , wherein a center of the warp operation performed on an aligned image is based on an overlapping portion of the image.
5. The method of claim 1 , wherein aligning overlapping portions of corresponding images comprises aligning an object of interest within overlapping portions of the corresponding images.
6. The method of claim 1 , wherein the warp operation is performed on the entire aligned image.
7. The method of claim 1 , wherein the warp operation is performed on a subset of the aligned image.
8. The method of claim 1 , wherein the warp operation maps the aligned image to a rectilinear matrix.
9. The method of claim 1 , wherein the warp operation removes distortion in the aligned image.
10. The method of claim 1 , wherein cropping the combined image comprises removing portions of the image that extend past a largest rectangular that can be fully contained within the combined image.
11. A system comprising at least one processor and a non-transitory computer readable medium comprising instructions configured to, when executed by one or more hardware processors, perform steps comprising:
capturing a plurality of images with each camera in a camera array comprising a plurality of cameras, each image comprising at least one portion overlapping with a corresponding portion of a corresponding image;
aligning overlapping portions of corresponding images to produce a set of aligned images;
for each aligned image, performing a warp operation on the aligned image to produce a warped image, wherein a magnitude of the warp operation on a portion of the aligned image increases with an increase in distance from the portion of the aligned image to the nearest overlapping portion of the aligned image;
taping each warped image together based on the overlapping portions of the warped images to form a combined image; and
cropping the combined image to produce a rectangular final image.
12. The system of claim 11 , wherein the plurality of images is captured synchronously.
13. The system of claim 11 , wherein the warp operation comprises a plurality of local warps performed on portions of each aligned image.
14. The system of claim 11 , wherein a center of the warp operation performed on an aligned image is based on an overlapping portion of the image.
15. The system of claim 11 , wherein aligning overlapping portions of corresponding images comprises aligning an object of interest within overlapping portions of the corresponding images.
16. The system of claim 11 , wherein the warp operation is performed on the entire aligned image.
17. The system of claim 11 , wherein the warp operation is performed on a subset of the aligned image.
18. The system of claim 11 , wherein the warp operation maps the aligned image to a rectilinear matrix.
19. The system of claim 11 , wherein the warp operation removes distortion in the aligned image.
20. The system of claim 11 , wherein cropping the combined image comprises removing portions of the image that extend past a largest rectangular that can be fully contained within the combined image.