Systems and methods for synthesizing high resolution images using images captured by an array of independently controllable imagers

1. A method for generating an image of a scene using an imager array including a plurality of imagers that each capture an image of the scene, photometric calibration data for the imager array, geometric calibration data for the imager, the method comprising:

obtaining input images captured by the plurality of imagers using a processor configured by image processing pipeline software, where the input images capture a scene in which depths of points in the imaged scene vary and each of the input images differs from the other input images due to scene dependent geometric displacements due to parallax experienced by each of the plurality of imagers based upon the different depths of the points in the imaged scene;

applying scene independent geometric corrections to the plurality of images using the geometric calibration data to obtain a plurality of geometrically registered images using the processor configured by image processing pipeline software;

determining scene dependent parallax information with respect to the input images based upon disparity relative to a reference point of view resulting from the different depths of points in the imaged scene using the processor configured by the image processing pipeline software, where the scene dependent parallax information comprises scene dependent geometric transformations;

determining an initial estimate of at least a portion of an image from a plurality of pixels from the plurality of input images based upon a total shift for each of the plurality of pixels relative to a reference view, where the total shift of a given pixel location is the combination of the scene independent geometric correction determined for the given pixel using the geometric calibration data and the scene dependent geometric transformation determined for the given pixel location; and

synthesizing an image using the initial estimate of the portion of the image.

2. The method of claim 1 , further comprising photometrically normalizing each of the plurality of captured images to obtain a plurality of normalized images and storing the plurality of normalized images in memory using the processor configured by image processing pipeline software.

3. The method of claim 2 , wherein the image processing pipeline software directs the processor to photometrically normalize each of the captured images using the photometric calibration data to obtain input images by performing at least one photometric image processing process selected from the group consisting of: photometric normalization; Black Level calculation and adjustments; vignetting correction; lateral color correction; and temperature normalization.

4. The method of claim 1 , wherein the imager array captures a single channel of information.

5. The method of claim 1 , wherein the imager array captures three channels of information.

6. The method of claim 5 , wherein the imager array includes imagers that sense red light (R), imagers that sense green light (G), and imagers that sense blue light (B).

7. The method of claim 1 , wherein the imager array captures four channels of information.

8. The method of claim 7 , wherein the four channels of information are the three RGB channels and a fourth near-IR channel.

9. The method of claim 8 , wherein the imager array includes imagers that sense red light (R), imagers that sense green light (G), imagers that sense blue light (B), and imagers that sense near-IR wavelengths.

10. The method of claim 1 , wherein the image processing pipeline software directs the processor to encode the synthesized image in accordance with the JPEG standard.

11. The method of claim 10 , wherein the image processing pipeline software directs the processor to store the encoded image in a file format selected from the group consisting of the JPEG Interchange Format (JIF), the JPEG File Interchange Format (JFIF), or the Exchangeable image file format (Exif).

12. The array camera of claim 1 , wherein imagers in the plurality of imagers have different fields of view resulting in a change in magnification of the image captured by the underlying sensor of the imager.

13. The method of claim 1 , wherein the image processing pipeline software directs the processor to generate a depth map for the synthesized image.

14. The method of claim 13 , wherein generating the depth map further comprises:

determining depth information for pixels in the synthesized image based upon the input images, parallax information, and the characteristics of the imager array; and

interpolating the depth information to obtain depth information for every pixel in the synthesized image.

15. The method of claim 1 , further comprising:

generating a focus map for the synthesized image; and

performing dynamic refocus of the synthesized image by rendering the synthesized image using the focus map.

16. The method of claim 15 , wherein the focus map identifies pixels having depths in the depth map that are within a specified depth of a defined focal plane.

17. The method of claim 1 , wherein the input images capture a scene in which depths of points in the imaged scene vary and each of the input images differs from the other input images due to:

scene independent geometric distortions inherent to the optics and manufacturing processes used to fabricate each of the plurality of imagers; and

scene dependent geometric displacements due to parallax based upon the depths of the points in the imaged scene.

18. The method of claim 17 , wherein:

the depths of points in the imaged scene vary due to the presence of foreground and background objects;

each of the input images also differ from the other input images due to occlusion zones surrounding foreground objects; and

the scene dependent parallax information also includes occlusion maps.

19. The method of claim 1 , further comprising fusing at least portions of the input images to form the initial estimate of at least one portion of the image by:

populating a grid corresponding to the pixel locations of the at least a portion of the initial estimate of the image with pixels from the geometrically registered input images using the scene dependent parallax information; and

interpolating the grid to obtain filtered pixel values for each pixel in the initial estimate of the image.

20. The method of claim 19 , wherein synthesizing an image using the initial estimate of at least a portion of the image comprises transforming, comparing and refining estimates until the at least one predetermined criterion is satisfied.

21. The method of claim 1 , wherein synthesizing an image using the initial estimate of at least a portion of the image comprises:

transforming the initial estimate of at least a portion of the image using at least one forward imaging transformation;

comparing the transformed initial estimate of at least a portion of the image to at least a portion of at least one geometrically registered input image; and

refining the estimate of the image based upon the comparison.

22. The method of claim 1 , wherein synthesizing an image using the initial estimate of at least a portion of the image comprises:

identifying pixels in the initial estimate of at least a portion of the image corresponding to pixels in at least one geometrically registered input image using at least one forward imaging transformation;

comparing the corresponding pixels; and

refining the estimate of the image based upon the comparison.

23. The method of claim 1 , wherein synthesizing an image using the initial estimate of at least a portion of the image comprises:

generating an estimate of at least a portion of the image; and

applying an intra-channel prior filter to the estimate of at least a portion of the image, where the intra-channel prior filter is configured to preserve edges while removing noise.

24. The method of claim 1 , wherein the imager array is an N×M array of imagers.

25. The method of claim 1 , wherein:

the imager array captures images in multiple color channels; and

the initial estimate of at least a portion of an image is an initial estimate of at least a portion of an image in a first color channel.