Capturing and processing of images using monolithic camera array with heterogeneous imagers
View Patent ↗A camera array, an imaging device and/or a method for capturing image that employ a plurality of imagers fabricated on a substrate is provided. Each imager includes a plurality of pixels. The plurality of imagers include a first imager having a first imaging characteristics and a second imager having a second imaging characteristics. The images generated by the plurality of imagers are processed to obtain an enhanced image compared to images captured by the imagers. Each imager may be associated with an optical element fabricated using a wafer level optics (WLO) technology.
1. A camera array, comprising:
a plurality of cameras forming an array comprising a plurality of rows of cameras and a plurality of columns of cameras, where each camera includes separate optics, and a plurality of sensor elements, and each camera is configured to independently capture an image of a scene;
wherein the optics of each camera are configured so that each camera has a field of view of the same scene that is shifted with respect to the field-of-views of the other cameras so that each shift includes a sub-pixel shifted view of the same scene;
wherein the camera array is a monolithic integrated module comprising:
a single semiconductor substrate on which all of the sensor elements and control circuitry are formed, where the control circuitry is configured to enable each camera to operate independently; and
optics including a plurality of lens elements, where each lens element forms part of the separate optics for one of the cameras; and
wherein the plurality of cameras comprises a plurality of different types of camera, where cameras of a given type share the same imaging characteristics and different types of cameras have different imaging characteristics including using different types of filters; and
wherein one of the plurality of cameras includes a central camera and the plurality of cameras forms:
at least one row of cameras above the central camera;
at least one row of cameras below the central camera;
at least one column of cameras to the left of the central camera; and
at least one column of cameras to the right of the central camera;
wherein cameras having the same imaging characteristics and filter are located in locations selected from the group consisting of:
locations including at least one row in the camera array above the central camera and at least one row in the camera array below the central camera; and
locations including at least one column in the camera array to the left of the central camera and at least one column in the camera array to the right of the central camera.
2. The camera array of claim 1 , wherein at least one of each different type of camera is located in each quadrant of the camera array.
3. The camera array of claim 2 , wherein the cameras of a given type of camera are symmetrically distributed about the central camera.
4. The camera array of claim 2 , wherein
each camera includes either a Blue filter, a Green filter, or a Red filter.
5. The camera array of claim 4 , wherein:
green cameras are located in locations including:
at least one row in the camera array above the central camera;
at least one row in the camera array below the central camera;
at least one column in the camera array to the left of the central camera; and
at least one column in the camera array to the right of the central camera.
6. The camera array of claim 5 , wherein:
red cameras are located in locations including at least one row in the camera array above the central camera and at least one row in the camera array below the central camera; and
blue cameras are located in locations including at least one column in the camera array to the left of the central camera and at least one column in the camera array to the right of the central camera.
7. The camera array of claim 5 , wherein:
blue cameras are located in locations including at least one row in the camera array above the central camera and at least one row in the camera array below the central camera; and
red cameras are located in locations including at least one column in the camera array to the left of the central camera and at least one column in the camera array to the right of the central camera.
8. The camera array of claim 2 , wherein
each camera includes either a Blue filter, a Green filter, a Red filter, or a near-IR filter.
9. The camera array of claim 1 , wherein each camera includes a filter selected from the group consisting of a Bayer filter, one or more Blue filters, one or more Green filters, one or more Red filters, one or more shifted spectral filters, one or more near-IR filters, and one or more hyper-spectral filters.
10. The camera array of claim 1 , wherein different types of cameras are configured by the control circuitry to operate with at least one difference in operating parameters.
11. The camera array of claim 10 , wherein the at least one difference in operating parameters includes at least one imaging parameter selected from the group consisting of exposure time, gain, and black level offset.
12. The camera array of claim 10 , wherein the control circuitry is configured to stagger the start of the exposure periods for sensor elements in different cameras so that a scene is sampled sequentially while having the sensor elements in several cameras exposed to light at the same time.
13. The camera array of claim 1 , wherein the camera array is at least a 4×4 array of cameras.
14. The camera array of claim 13 , wherein the camera array is a 4×4 array of cameras.
15. The camera array of claim 13 , wherein the camera array is a 5×5 array of cameras.
16. The camera array of claim 13 , wherein the camera array is a 6×6 array of cameras.
17. The camera array of claim 1 , wherein:
the plurality of cameras includes different types of cameras dedicated to sampling luma and dedicated to sampling chroma; and
cameras that sample chroma are arranged symmetrically within the camera array to address occlusion due to parallax.
18. The camera array of claim 1 , wherein the central camera is a polychromatic camera.
19. The camera array of claim 18 , wherein:
the plurality of cameras also includes a plurality of cameras dedicated to sampling chroma; and
cameras that sample chroma are arranged symmetrically within the camera array to address occlusion due to parallax.
20. A linear camera array, comprising:
a plurality of cameras forming a linear array, where each camera includes separate optics, and a plurality of sensor elements, and each camera is configured to independently capture an image of a scene;
wherein the optics of each camera are configured so that each camera has a field of view that is shifted with respect to the field-of-views of the other cameras so that each shift includes a sub-pixel shifted view of the scene;
wherein the linear camera array is a monolithic integrated module comprising:
a single semiconductor substrate on which all of the sensor elements and control circuitry are formed, where the control circuitry is configured to enable each camera to operate independently; and
optics including a plurality of lens elements, where each lens element forms part of the separate optics for one of the cameras; and
wherein the plurality of cameras comprises a plurality of different types of camera, where cameras of a given type share the same imaging characteristics and different types of cameras have different imaging characteristics including using different types of filters; and
wherein one of the plurality of cameras forms a central camera;
wherein cameras having the same imaging characteristics and filter are located on either side of the central camera.
21. The camera array of claim 20 , wherein the cameras that include a specific type of filter are symmetrically distributed about the central camera.
22. The camera array of claim 20 , wherein
each camera includes either a Blue filter, a Green filter, or a Red filter.
23. The camera array of claim 20 , wherein
each camera includes either a Blue filter, a Green filter, a Red filter, or a near-IR filter.
24. The camera array of claim 20 , wherein each camera includes a filter selected from the group consisting of a Bayer filter, one or more Blue filters, one or more Green filters, one or more Red filters, one or more shifted spectral filters, one or more near-IR filters, and one or more hyper-spectral filters.
25. The camera array of claim 20 , wherein different types of cameras operate with at least one difference in operating parameters.
26. The camera array of claim 25 , wherein the at least one difference in operating parameters includes at least one imaging parameter selected from the group consisting of exposure time, gain, and black level offset.
27. The camera array of claim 25 , wherein the control circuitry is configured to stagger the start of the exposure periods for sensor elements in different cameras so that a scene is sampled sequentially while having the sensor elements in several cameras exposed to light at the same time.
28. The camera array of claim 20 , wherein:
the plurality of cameras includes different types of cameras dedicated to sampling luma and dedicated to sampling chroma; and
cameras that sample chroma are arranged symmetrically within the camera array to address occlusion due to parallax.
29. The camera array of claim 20 , wherein the central camera is a polychromatic camera.
30. The camera array of claim 29 , wherein:
the plurality of cameras also includes a plurality of cameras dedicated to sampling chroma; and
cameras that sample chroma are arranged symmetrically within the camera array to address occlusion due to parallax.