Imaging system with foveated imaging capabilites
View Patent ↗An electronic device may have a camera module. The camera module may include a camera sensor capable of capturing foveated images. The camera sensor may be hardwired to capture foveated images with fixed regions of different quality levels or may be dynamically-reconfigurable to capture foveated images with selected regions of different quality levels. As one example, the camera module may be hardwired to capture a center region of an image at full resolution and peripheral regions at reduced resolutions, so that a user can merely center objects of interest in the image to capture a foveated image. As another example, the camera module may analyze previous images to identify objects of interest and may then reconfigure itself to capture the identified objects of interest at a high quality level, while capturing other regions at reduced quality levels.
1. A system, comprising:
a central processing unit;
memory;
input-output circuitry; and
an imaging device, wherein the imaging device comprises:
an array of image sensing pixels divided into at least first and second regions, wherein each image sensing pixel comprises a photosensitive element;
a transistor connected to each of the photosensitive elements in a given pair of image sensing pixels;
address generator circuitry coupled to the array of image sensing pixels; and
image readout circuitry coupled to the array of image sensing pixels, wherein the image readout circuitry is operable to readout image data from the first region of the array at a first resolution and is operable to readout image data from the second region of the array at a second resolution, wherein the first resolution is greater than the second resolution, and wherein the first and second resolutions are based on control signals from the address generator circuitry applied to the transistor.
2. The system defined in claim 1 wherein the image sensing pixels of the array are formed in a uniform density across the array.
3. The system defined in claim 2 wherein the array of image sensing pixels comprises conductive paths between pairs of image sensing pixels in the second region of the array, wherein each conductive path is operable to bin together one of the pairs of image sensing pixels in the second region of the array, and wherein the image sensing pixels in the first region of the array are not binned together.
4. The system defined in claim 1 wherein the image sensing pixels in the first region of the array are formed in a first density, wherein the image sensing pixels in the second region of the array are formed in a second density, and wherein the first density is greater than the second density.
5. The system defined in claim 1 wherein the image readout circuitry is operable to readout image data from first region at the first resolution and from the second region at the second resolution as part of capturing a first image and wherein, as part of capturing a second image, the image readout circuitry is operable to readout image data from the first region at the second resolution and is operable to readout image data from the second region at the first resolution.
6. The system defined in claim 5 wherein the image sensing pixels of the array are formed in a uniform density across the array.
7. The system defined in claim 1 wherein the imaging device further comprises image processing circuitry operable to generate a foveated image by combining the image data from the first and second regions of the array.
8. A method of using an image sensor that includes first and second pluralities of image sensing pixels and address generator circuitry coupled to the first and second pluralities of image sensing pixels, wherein each image sensing pixel comprises a photosensitive element, and wherein a transistor is connected to each of the photosensitive elements in a given pair of image sensing pixels, the method comprising:
capturing a first image portion at a first resolution using the first plurality of image sensing pixels;
capturing a second image portion at a second resolution using the second plurality of image sensing pixels, wherein the first resolution is greater than the second resolution and wherein the first and second resolutions are based on control signals from the address generator circuitry applied to the transistor; and
combining the first and second image portions into a foveated image.
9. The method defined in claim 8 wherein the first plurality of image sensing pixels are spread across the image sensor in a first density, wherein the second plurality of image sensing pixels are spread across the image sensor in a second density, and wherein the first density is greater than the second density.
10. The method defined in claim 8 wherein the first and second pluralities of image sensing pixels are spread across the image sensor in a common density and wherein capturing the second image portion at the second resolution comprises binning together pixel charges from at least some of the image sensing pixels in the second plurality of image sensing pixels.
11. The method defined in claim 8 further comprising:
capturing a third image portion at a given resolution using the first plurality of image sensing pixels;
capturing a fourth image portion at the given resolution using the second plurality of image sensing pixels; and
combining the third and fourth image portions into a non-foveated image having a constant resolution across the non-foveated image.
12. The method defined in claim 8 further comprising:
capturing a third image portion at the second resolution using the first plurality of image sensing pixels;
capturing a fourth image portion at the first resolution using the second plurality of image sensing pixels; and
combining the third and fourth image portions into an additional foveated image.
13. A method comprising:
using an image sensor comprising a plurality of image sensing pixels and address generator circuitry coupled to the plurality of image sensing pixels, wherein each image sensing pixel comprises a photosensitive element, and wherein a transistor is connected to each of the photosensitive elements in a given pair of image sensing pixels, capturing an image of a scene;
using image processing circuitry and the image, identifying at least a first point of interest in a first region of the scene; and
configuring the image sensor to capture a foveated image in which the first region of the scene is captured at a first resolution and in which at least a second region of the scene is captured at a second resolution that is less than the first resolution, and wherein the first and second resolutions are based on control signals from the address generator circuitry applied to the transistor.
14. The method defined in claim 13 wherein configuring the image sensor to capture the foveated image comprises applying binning control signals to image sensing pixels that are associated with the second region of the scene.
15. The method defined in claim 13 wherein the foveated image is one of a plurality of foveated images in a video stream, the method further comprising:
creating the video stream of foveated images using the image sensor and the image processing circuitry.
16. The method defined in claim 15 wherein creating the video stream of foveated images comprises:
capturing a given frame that includes the second region of the scene;
capturing a plurality of additional frames that each include the first region of the scene; and
forming each of the plurality of foveated images by combining the given frame with a respective one of the additional frames.
17. The method defined in claim 13 wherein identifying the first point of interest in the first region comprises detecting a face in the image.
18. The method defined in claim 13 wherein the first point of interest in the first region of the scene comprises an object of interest in the first region of the scene, the method further comprising:
using the image processing circuitry and the foveated image, determining that the object of interest has moved into the second region of the foveated image;
reconfiguring the image sensor to capture an additional foveated image in which the second region of the scene is captured at the first resolution and in which at least the first region of the scene is captured at the second resolution.