Systems and methods for estimating depth and visibility from a reference viewpoint for pixels in a set of images captured from different viewpoints

1. An array camera, comprising:

a plurality of cameras;

a processor;

memory containing an image processing application and calibration data;

wherein the processor is configured by the image processing application to produce an image by:

directing the plurality of cameras to capture a plurality of images including a reference image and at least one alternate view image;

retrieving the calibration data from memory;

rectifying the plurality of captured images using the calibration data;

performing a calibration process;

determining a range of disparities to search based upon characteristics of at least one of the plurality of images;

measuring similarity of corresponding pixels in the plurality of images at a plurality of disparities;

generating an initial depth map; and

refocusing the reference image using the initial depth map;

wherein the processor is configured by the image processing application to measure similarity of corresponding pixels in the plurality of images at the plurality of disparities using a cost function.

2. The array camera of claim 1 , wherein the calibration process comprises: obtaining additional scene-independent geometric corrections that compensate for scene-independent geometric distortions that remain after rectification using offline the calibration data.

3. The array camera of claim 2 , wherein scene-independent geometric corrections determined using rectification are determined at a sub-pixel resolution.

4. The array camera of claim 1 , wherein the processor is configured by the image processing application to:

determine characteristics of at least one of the plurality of images;

determine the range of disparities to search based upon the determined characteristics.

5. The array camera of claim 1 , wherein the processor is configured by the image processing application to select a set of disparities to search across the range of disparities.

6. The array camera of claim 5 , wherein the set of disparities do not provide uniform sampling of disparity.

7. The array camera of claim 1 , wherein the processor is configured by the image processing application to select a set of disparities to search across the range of disparities based upon prior information about where objects are in the at least one of the plurality of images.

8. The array camera of claim 1 , wherein the processor is configured by the image processing application to measure similarity of corresponding pixels using a block-based similarity measure.

9. The array camera of claim 1 , wherein the processor is configured by the image processing application to filter calculated costs using an edge preserving filter.

10. The array camera of claim 9 , wherein the processor is configured by the image processing application to filter the calculated costs using a bilateral filter.

11. The array camera of claim 10 , wherein the processor is configured by the image processing application to determine filter weights of the bilateral filter based upon the reference image and the resulting filter weights are applied to the calculated costs.

12. The array camera of claim 1 , wherein the processor is configured by the image processing application to identify pixels that may be occluded using the initial depth map by identifying pixel depth values that are inconsistent with local pixel depth values.

13. The array camera of claim 1 , wherein the processor is configured by the image processing application to identify occluded pixels based upon scene-dependent geometric shifts of the pixels.

14. The array camera of claim 1 , wherein the processor is configured by the image processing application to generate a depth of an occluded pixel by averaging depth of surrounding pixels.

15. The array camera of claim 1 , wherein the processor is configured by the image processing application to generate a depth of an occluded pixel by interpolating depths of adjacent pixels for which depths have been calculated.

16. The array camera of claim 1 , wherein the processor is configured by the image processing application to generate a depth estimate for a given pixel location in the initial depth map by:

identifying at least one pixel in the plurality of images that corresponds to the given pixel location in the reference image based upon expected disparity at the plurality of depths;

comparing the similarity of the pixels that are identified as corresponding at each of the plurality of depths; and

selecting a depth from the plurality of depths at which the identified corresponding pixels have a highest degree of similarity as a depth estimate for the given pixel location.

17. The array camera of claim 16 , wherein the processor is configured by the image processing application to select the depth from the plurality of depths at which the identified corresponding pixels have the highest degree of similarity as the depth estimate for the given pixel location by selecting the depth from the plurality of depths at which a spatially filtered cost function for the identified corresponding pixels indicates the highest level of similarity.

18. The array camera of claim 16 , wherein the processor is configured by the image processing application to compare the similarity of the pixels that are identified as corresponding at each of the plurality of depths by comparing intensity values of corresponding pixels across color channels.