Extended color processing on pelican array cameras

1. A method of generating an image of a scene using a camera array including at least one camera that captures an RGB image of a scene and at least one camera that captures near-infrared (IR) spectral wavelengths of the scene, the method comprising:

obtaining input images captured by a plurality of cameras that includes a camera that captures an RGB image and a camera that captures near-IR wavelengths, where the input images includes a first input image that includes image information captured in at least three channels (RGB) of information and a second input image that includes image information captured in at least a near-IR channel of information;

generate a fused image using a processor configured by software to:

measure parallax using the input images captured by the plurality of cameras to produce a depth map;

normalize the second input image in the photometric reference space of the first input image;

cross-channel normalize the first input image with respect to the second input image by applying gains and offsets to pixels of the first input image; and

perform cross-channel fusion using the first input image and the second input image to produce an image.

2. The method of claim 1 , wherein the first input image and the second input image have the same resolution.

3. The method of claim 1 , further comprising:

capturing a first set of input images that are captured by a first set of RGB cameras from the plurality of cameras; and

capturing a second set of input images that are captured by a second near-IR camera from the plurality of cameras.

4. The method of claim 3 , further comprising:

combining image information from the first set of input images into a first fused image using analog gain and noise information from the first set of RGB cameras; and

combining image information from the second set of input images into a second fused image utilizes using analog gain and noise information from the second near-IR camera.

5. The method of claim 1 , further comprising denoising the first input image using a first bilateral filter and denoising the second input image using a second bilateral filter, wherein the first bilateral filter and the second bilateral filter utilize weights that are a function of both the photometric and geometric distance between a pixel and pixels in the neighborhood of the pixel.

6. The method of claim 5 , wherein the first bilateral filter is a cross-channel bilateral filter utilizing weights determined for the second input image.

7. The method of claim 5 , wherein the first input image is captured by a first camera from the plurality of cameras and the first bilateral filter is a cross-channel bilateral filter utilizing weights determined for the second input image when an analog gain value of the first camera is above a predetermined threshold.

8. The method of claim 1 , wherein normalizing the second input image in the photometric reference space of the first input image comprises applying gains and offsets to pixels of the second input image.

9. The method of claim 1 , wherein the processor being configured to normalize the second input image in the photometric reference space of the first input image comprises the processor being configured to:

select a first pixel of interest in the second input image and a first collection of similar pixels in the neighborhood of the first pixel of interest;

select a second pixel of interest in the first input image corresponding to the first pixel of interest and a second collection of similar pixels in the neighborhood of the second pixel of interest;

determine the intersection of the first collection of similar pixels and the second collection of similar pixels;

calculate gain and offset values using the intersection of the two collections;

apply the gain and offset values to the appropriate pixels in the second input image.

10. The method of claim 9 , where the intersection of the first collection of similar pixels and the second collection of similar pixels is the set of pixels in the first and second collections having the same corresponding locations in each of the first input image and the second input image.

11. An array camera configured to generate an image of a scene using an array camera including at least one camera that captures an RGB image of a scene and at least one camera that captures at least near-infrared (IR) spectral wavelengths of the scene, the array camera comprising:

an array camera including a plurality of cameras that includes a camera that captures an RGB image and a camera that captures at least near-IR spectral wavelengths; and

a processor configured by software to:

obtain input images captured by the plurality of cameras that includes the camera that captures an RGB image and the camera that captures at least near-IR spectral wavelengths, where the input images includes a first input image that includes image information captured in at least three channels (RGB) of information and a second input image that includes image information captured in a near-IR channel of information;

generate a fused image by:

measuring parallax using the input images captured by the plurality of cameras to produce a depth map;

normalizing the second input image in the photometric reference space of the first input image;

cross-channel normalize the first input image with respect to the second input image by applying gains and offsets to pixels of the first input image; and

perform cross-channel fusion using the first input image the second input image to produce an image.

12. The array camera of claim 11 , wherein the three channels (RGB) of information includes green and red light.

13. The array camera of claim 11 , wherein the first input image and the second input image have the same resolution.

14. The array camera of claim 11 , wherein the processor is further configured to:

capture a first set of input images that are captured by a first set of RGB cameras from the plurality of cameras; and

capture a second set of input images that are captured by a second near-IR camera from the plurality of cameras.

15. The array camera of claim 14 , wherein the processor is further configured to:

combine image information from the first set of input images into a first fused image using analog gain and noise information from the first set of RGB cameras; and

combine image information from the second set of input images into a second fused image utilizes using analog gain and noise information from the second near-IR camera.

16. The array camera of claim 11 , wherein the processor is further configured to denoise the first input image using a first bilateral filter and denoise the second input image using a second bilateral filter, wherein the first bilateral filter and the second bilateral filter utilize weights that are a function of both the photometric and geometric distance between a pixel and pixels in the neighborhood of the pixel.

17. The array camera of claim 16 , wherein the first bilateral filter is a cross-channel bilateral filter utilizing weights determined for the second input image.

18. The array camera of claim 16 , wherein the first input image is captured by a first camera from the plurality of cameras and the first bilateral filter is a cross-channel bilateral filter utilizing weights determined for the second input image when an analog gain value of the first camera is above a predetermined threshold.

19. The array camera of claim 11 , wherein normalizing the second input image in the photometric reference space of the first input image comprises applying gains and offsets to pixels of the second input image.