Method and apparatus for generating depth map from monochrome microlens and imager arrays
A method and apparatus for creating an adaptive mosaic pixel-wise virtual Bayer pattern. The method may include receiving a plurality of monochromatic images from an array of imaging elements, creating a reference ordered set at infinity from the plurality of monochromatic images, running a demosaicing process on the reference ordered set, and creating a color image from the demosaiced ordered set. One or more offset artifacts resulting from the demosaicing process may be computed at a distance other than infinity, the ordered set may be modified in accordance with the computed offsets, and a demosaiced image may be regenerated in accordance with modified ordered set. The regenerating process may be repeated for one or more of the one or more offsets.
1. A method for creating an adaptive mosaic pixel-wise virtual Bayer pattern, comprising the steps of:
receiving a plurality of monochromatic images from an array of imaging elements;
creating a reference ordered set at infinity from the plurality of monochromatic images;
running a demosaicing process on the reference ordered set;
creating a color image from the demosaiced ordered set;
computing one or more offset artifacts resulting from the demosaicing process at a distance other than infinity;
modifying the ordered set in accordance with the computed offsets;
regenerating a demosaiced image in accordance with modified ordered set; and
repeating the regenerating process for one or more of the one or more offsets.
2. The method of claim 1 , wherein the adaptive Bayer pattern is updated by matching row-wise and column wise disparity values.
3. The method of claim 1 , further comprising the steps of:
realigning the reference image such that objects that are not at infinity are subtended by a modified Bayer pattern for better image quality;
creating a second pattern based upon the modified Bayer pattern; and
generating a second demosaiced image in accordance with the second pattern.
4. The method of claim 1 , wherein a hand of a user is included in the monochromatic images.
5. The method of claim 4 , wherein the hand of the user is used to control one or more devices.
6. The method of claim 5 , wherein the one or more devices comprises a camera.
7. The method of claim 6 , wherein the hand of the user is positioned within a field of view of the camera.
8. The method of claim 6 , wherein the array of imaging elements is backward facing and the hand of the user is positioned behind a display of a camera within the field of view of the array of imaging elements.
9. The method of claim 8 , wherein a pinch gesture by the hand of the user adjusts zoom on the display of the camera without requiring contact with the display of the camera.
10. The method of claim 8 , wherein a thumbs up gesture results in the taking of a picture by the camera.
11. The method of claim 8 , wherein movement of the hand of the user towards and away from the display of the camera adjusts zoom of the camera.
12. The method of claim 5 , wherein the one or more devices comprise one or more electronic devices.
13. The method of claim 12 , wherein the one or more electronic devices are controlled by a gestural interface, each gesture being determined based upon one or more images acquired by the array of imaging elements.
14. The method of claim 12 , wherein the one or more electronic devices comprises a television.
15. The method of claim 12 , wherein the one or more devices comprises a game console.
16. The method of claim 1 , wherein the step of computing one or more offset artifacts is performed only on pixels in the reference image that have changed from a prior image.
17. The method of claim 1 , further comprising the steps of:
highlighting on a touchscreen, by the user, one or more regions in the color image;
extracting the coordinates of the highlighted region from one or more of the color image and the set of monochromatic images;
segmenting the region in the one of the set of monochromatic images and the color image from which the coordinates were extracted,
determining a distance to the highlighted region on the display;
determining on or more dimensions of the highlighted region in accordance with the demosaicing process.
18. The method of claim 17 , wherein the highlighted region is defined by detection of an object on the touchscreen.
19. A system for creating an adaptive mosaic pixel-wise virtual Bayer pattern, comprising:
an array of imaging elements for acquiring a plurality of monochromatic images; and
a processor for creating a reference ordered set at infinity from the plurality of monochromatic images, running a demosaicing process on the reference ordered set, creating a color image from the demosaiced ordered set, computing one or more offset artifacts resulting from the demosaicing process at a distance other than infinity, modifying the ordered set in accordance with the computed offsets, regenerating a demosaiced image in accordance with modified ordered set, and repeating the regenerating process for one or more of the one or more offsets.
20. A non-transitory storage medium having a computer program stored thereon, the computer program causing a general purpose computer to perform the steps of:
receiving a plurality of monochromatic images from an array of imaging elements;
creating a reference ordered set at infinity from the plurality of monochromatic images;
running a demosaicing process on the reference ordered set;
creating a color image from the demosaiced ordered set;
computing one or more offset artifacts resulting from the demosaicing process at a distance other than infinity;
modifying the ordered set in accordance with the computed offsets;
regenerating a demosaiced image in accordance with modified ordered set; and
repeating the regenerating process for one or more of the one or more offsets.
21. The non-transitory storage medium of claim 19 , wherein the general purpose computer further comprises a graphical processing unit.