Image taping in a multi-camera array
Multiple cameras are arranged in an array at a pitch, roll, and yaw that allow the cameras to have adjacent fields of view such that each camera is pointed inward relative to the array. The read window of an image sensor of each camera in a multi-camera array can be adjusted to minimize the overlap between adjacent fields of view, to maximize the correlation within the overlapping portions of the fields of view, and to correct for manufacturing and assembly tolerances. Images from cameras in a multi-camera array with adjacent fields of view can be manipulated using low-power warping and cropping techniques, and can be taped together to form a final image.
1. A method comprising:
capturing a plurality of images with each camera in a 2×2 array of cameras, each image comprising a first portion representative of an overlapping field of view with a corresponding portion of a horizontally adjacent image and a second portion representative of an overlapping field of view with a corresponding portion of a vertically adjacent image;
for each image, performing a first warping on the image such that the first portion of the image is substantially aligned with the corresponding portion of the horizontally adjacent image and such that the second portion of the image is substantially aligned with the corresponding portion of the vertically adjacent image;
for each image, resizing the image by cropping portions of the image that are not both horizontally aligned with the first portion of the image and vertically aligned with the second portion of the image;
for each image, performing a second warping on the image to reduce distortion resulting from the first warping, wherein a magnitude of the second warping of a portion of the image is based on a distance of the portion to the first portion of the image and the second portion of the image, wherein the magnitude of the second warping of the first portion of the image and the second portion of the image is substantially zero;
cropping portions of the plurality of images representative of overlapping fields of view of corresponding portions of adjacent images to substantially minimize the portions of the cropped images representative of overlapping fields of view of corresponding cropped images; and
generating a final image by taping each cropped image to a horizontally adjacent cropped image and a vertically adjacent cropped image.
2. The method of claim 1 , wherein the plurality of images is captured synchronously.
3. The method of claim 1 , wherein the first warping comprises a plurality of local warps on portions of the image.
4. The method of claim 1 , wherein a center of the first warping of an image is based on an overlapping field of view of the image.
5. The method of claim 1 , wherein the first warping substantially aligns an object of interest within each overlapping field of view.
6. The method of claim 1 , wherein the first warping performed on an image warps the entire image.
7. The method of claim 1 , wherein the first warping performed on an image warps a plurality of portions of the image.
8. The method of claim 1 , wherein the second warping of an image maps the image to a rectilinear matrix.
9. The method of claim 1 , wherein the second warping performed on an image removes distortion in the image while maintaining alignment of an object of interest within each overlapping field of view.
10. The method of claim 1 , wherein taping each cropped image to a horizontally adjacent cropped image and a vertically adjacent cropped image further comprises:
for each image, cropping the portions overlapping with corresponding portions of horizontally and vertically adjacent images; and
concatenating the images with the horizontally and vertically adjacent cropped image.
11. A method comprising:
capturing a plurality of images with each camera in a plurality of cameras in a camera array, each image comprising a field of view (FOV) portion representative of an overlapping field of view with at least one corresponding FOV portion of an adjacent image;
for each image, performing a first warping on the image such that the FOV portion of the image is substantially aligned with the corresponding FOV portion of the adjacent image;
for each image, resizing the image by cropping portions of the image that are not aligned with the FOV portion of the image;
for each image, performing a second warping on the image to reduce distortion resulting from the first warping, wherein a magnitude of the second warping of a portion of the image is based on a distance of the portion to the FOV portion of the image, wherein the magnitude of the second warping of the FOV portion of the image is substantially zero;
cropping portions of the plurality of images representative of FOV portions overlapping with corresponding FOV portions of adjacent images; and
generating a final image by taping each cropped image to an adjacent cropped image.
12. The method of claim 11 , wherein the plurality of images is captured synchronously.
13. The method of claim 11 , wherein the first warping comprises a plurality of local warps on portions of the image.
14. The method of claim 11 , wherein a center of the first warping of an image is based on a portion in a FOV portion of the image.
15. The method of claim 11 , wherein the first warping substantially aligns an object of interest within a FOV portion of the image.
16. The method of claim 11 , wherein the first warping performed on an image warps the entire image.
17. The method of claim 11 , wherein the first warping performed on an image warps a plurality of portions of the image.
18. The method of claim 11 , wherein the second warping of an image maps the image to a rectilinear matrix.
19. The method of claim 11 , wherein the second warping performed on an image removes distortion in the image while maintaining alignment of an object of interest within a FOV portion.
20. The method of claim 11 , wherein taping each cropped image to an adjacent cropped image further comprises:
for each image, cropping the FOV portions; and
concatenating the images with the adjacent cropped image.
21. A system comprising at least one processor and a non-transitory computer readable medium comprising instructions for taping images captured by a plurality of cameras in a camera array, the instructions executable by the at least one processor, the instructions comprising instructions for:
capturing a plurality of images with each camera in a 2×2 array of cameras, each image comprising a first portion representative of an overlapping field of view with a corresponding portion of a horizontally adjacent image and a second portion representative of an overlapping field of view with a corresponding portion of a vertically adjacent image;
for each image, performing a first warping on the image such that the first portion of the image is substantially aligned with the corresponding portion of the horizontally adjacent image and such that the second portion of the image is substantially aligned with the corresponding portion of the vertically adjacent image;
for each image, resizing the image by cropping portions of the image that are not both horizontally aligned with the first portion of the image and vertically aligned with the second portion of the image;
for each image, performing a second warping on the image to reduce distortion resulting from the first warping, wherein a magnitude of the second warping of a portion of the image is based on a distance of the portion to the first portion of the image and the second portion of the image, wherein the magnitude of the second warping of the first portion of the image and the second portion of the image is substantially zero;
cropping portions of the plurality of images representative of overlapping fields of view of corresponding portions of adjacent images to substantially minimize the portions of the cropped images representative of overlapping fields of view of corresponding cropped images; and
generating a final image by taping each cropped image to a horizontally adjacent cropped image and a vertically adjacent cropped image.
22. The system of claim 21 , wherein the plurality of images is captured synchronously.
23. The system of claim 21 , wherein the first warping comprises a plurality of local warps on portions of the image.
24. The system of claim 21 , wherein a center of the first warping of an image is based on an overlapping field of view of the image.
25. The system of claim 21 , wherein the first warping substantially aligns an object of interest within each overlapping field of view.
26. The system of claim 21 , wherein the first warping performed on an image warps the entire image.
27. The system of claim 21 , wherein the first warping performed on an image warps a plurality of portions of the image.
28. The system of claim 21 , wherein the second warping of an image maps the image to a rectilinear matrix.
29. The system of claim 21 , wherein the second warping performed on an image removes distortion in the image while maintaining alignment of an object of interest within each overlapping field of view.
30. The system of claim 21 , wherein taping each cropped image to a horizontally adjacent cropped image and a vertically adjacent cropped image further comprises:
for each image, cropping the portions overlapping with corresponding portions of horizontally and vertically adjacent images; and
concatenating the images with the horizontally and vertically adjacent cropped image.