Array camera imaging system and method
View Patent ↗Aspects of the disclosed technology relate to an imaging system and method in which an array camera is employed along with an image processor to make use of parallax convergence differences between different cameras or groups of cameras within an array camera to provide improved super resolution performance.
1. An imaging device comprising:
an array camera having at least M×N cameras, the camera array configured to capture M×N images, where M and N are positive integers having a product of at least two; and
an image processor operatively coupled to the array camera, the image processor configured to:
parallax converge images from each camera of the array camera based on different predefined offset points within a predefined sampling pattern to create a sub-pixel sampling pattern; and
sample the sub-pixel sampling pattern at an increased resolution to provide a super resolution output image.
2. The imaging device of claim 1 , wherein the array camera includes an array of 4×4 cameras.
3. The imaging device of claim 1 , wherein the cameras within the array camera are monochromatic cameras arranged in groups.
4. The imaging device of claim 3 , wherein, for each group of cameras, the image processor is configured to parallax converge images from the group of cameras based on a different predefined offset point within the predefined sampling pattern, producing 4 group outputs.
5. The imaging device of claim 4 , wherein the image processor is configured to parallax converge the 4 group outputs based on the different predefined offset points.
6. The imaging device of claim 1 , wherein the increased resolution is at least a three times (3×) resolution.
7. The imaging device of claim 1 , wherein the image processor is configured to sample the sub-pixel sampling pattern using a base sample contribution falloff.
8. The imaging device of claim 7 , wherein the image processor is configured to sample the sub-pixel sampling pattern using a spherical sample contribution falloff sampling to surrounding samples.
9. The imaging device of claim 1 , wherein the predefined sampling pattern is a Hilbert sampling pattern.
10. The imaging device of claim 4 , wherein the different predefined offset points create vertical, horizontal and diagonal differences between the 4 group outputs.
11. The imaging device of claim 1 , wherein the predefined offset points include at least four points per pixel that are relatively offset vertically, horizontally and diagonally.
12. A portable communication device comprising the imaging device of claim 1 .
13. The portable communication device of claim 12 , wherein the portable communication device is a mobile phone.
14. A method of generating an image, the method comprising:
capturing image data using an array camera having at least N×N cameras, where N is at least two, the array camera producing N×N images, wherein the array camera includes N groups of N cameras;
for each group of N cameras of the array camera, parallax converging the image from the group of N cameras based on a different predefined offset point within a predefined sampling pattern, producing N group outputs;
parallax converging the N group outputs based on the different predefined offset points; and
creating a super resolution image by sampling each of the N group outputs based on the predefined offset points within the predefined sampling pattern at an increased resolution.
15. The method of claim 14 , wherein N=4 and the cameras within the array camera are monochromatic cameras.
16. The method of claim 14 , wherein the increased resolution is at least a three times (3×) resolution.
17. The method of claim 14 , wherein sampling includes using a base sample contribution falloff.
18. The method of claim 14 , wherein the different predefined offset points create vertical, horizontal and diagonal differences between the 4 group outputs.
19. The method of claim 14 , wherein the predefined offset points include at least four points per pixel offset vertically, horizontally and diagonally.
20. A method of generating a super resolution image, the method comprising:
capturing image data using an array camera having at least four cameras;
parallax converging image data from each camera of the array camera based on different predefined offset points within a predefined sampling pattern to create a sub-pixel sampling pattern; and
sampling the sub-pixel sampling pattern at an increased resolution.