Object detection using an in-sensor detector
View Patent ↗Systems and methods are provided for detecting an object of object class, such as faces, in an image sensor. In some embodiments, the image sensor can provide a scan sequence that scans a scene over multiple time intervals. The image sensor can scan, in succession, portions of a scene, where each of the portions covers a different amount or location of the scene. This way, the scanned portions can be saved in an image buffer that is sized significantly smaller than an entire frame. In some embodiments, when the image sensor detects the presence of an object of the object class, the image sensor can store positional information (e.g., location and size of the object) in a region of interest buffer. The image sensor can output the positional information to aid an electronic device, such as a camera, perform various functions, such as automatic exposure and color balancing.
1. A method of scanning a scene using an image sensor, the method comprising:
scanning a first portion of the scene in a first time interval;
storing the first portion in a buffer for image processing;
scanning a second portion of the scene in a second time interval subsequent to the first time interval, wherein the second portion covers a different amount or location of the scene than the first portion; and
saving, for image processing, the second portion in the buffer in place of the first portion.
2. The method of claim 1 , wherein:
scanning the first portion comprises scanning a first vertical stripe; and
scanning the second portion comprises scanning a second vertical stripe, wherein the first and second vertical stripes each has a width less than a width of the scene.
3. The method of claim 2 , wherein the first vertical stripe has a same width as the second vertical stripe, and wherein the first vertical stripe has a different horizontal offset as the second vertical stripe.
4. The method of claim 2 , wherein the first vertical stripe has a different width from a width of the second vertical stripe.
5. The method of claim 1 further comprising:
detecting the presence of objects in an object class in the first portion of the scene while the first portion is being stored in the buffer; and
detecting the presence of objects in the object class in the second portion of the scene while the second portion is being stored in the second buffer.
6. The method of claim 1 wherein storing the first or second portions of the scene comprises:
saving luminance information for the portion of the scene.
7. An image sensor comprising:
an image scanner configured to scan, in succession, portions of a scene, wherein each of the portions covers a different amount or location of the scene, wherein the succession of portions comprises:
a first plurality of vertical stripes with differing horizontal offsets, each having a first predetermined width; and
a second plurality of vertical stripes with differing horizontal offsets, each having a second predetermined width; and
an object detector configured to analyze at least one of the portions, while that portion is being scanned, for the presence of an object in an object class.
8. The image sensor of claim 7 , wherein the object class comprises faces.
9. The image sensor of claim 7 , further comprising:
a rolling buffer sized to store less than a smallest of the portions of the scene, wherein each of the portions is stored in the buffer by re writing previously stored data in a circular fashion.
10. The image sensor of claim 7 , wherein the rolling buffer comprises a plurality of partitions, and wherein the detector is configured to analyze data from a subset of the partitions while a remainder of the partitions is being filled.
11. The image sensor of claim 7 , further comprising:
a downsampler configured to sub-sample at least one of the portions and store the sub-sampled portions into the rolling buffer.
12. An image sensor comprising:
a first buffer sized to store at least a portion of a scene;
a second buffer; and
detection hardware configured to:
determine, from contents of the first buffer, regions in the scene that potentially contain at least one object of an object class; and
store positional information of the regions in the second buffer.
13. The image sensor of claim 12 , wherein the positional information comprise at least one of: location of the regions, size of the regions, and strength of detection.
14. The image sensor of claim 12 , wherein the detection hardware is configured to determine regions by:
picking a plurality of blocks of the scene, wherein the blocks are of a predetermined size; and
identifying whether each of the plurality of blocks includes at least one object of the object class.
15. The image sensor of claim 12 , further comprising:
an image scanner configured to determine which portion of the scene to store in the first buffer; and
a downsampler configured to downsample the determined portion of the scene.
16. The image sensor of claim 12 , further comprising:
a first output for outputting image data corresponding to the scene; and
a second output for outputting contents of the second buffer.
17. An electronic device comprising:
an image sensor configured to capture a scene in portions, the image sensor comprising:
a detector for detecting regions in the scene that potentially contain objects of an object class and
a buffer for storing positional information corresponding to the regions; and
control circuitry configured to obtain the positional information from the image sensor and perform an action based on the positional information.
18. The electronic device of claim 17 , wherein the buffer is a first-in first-out buffer.
19. The electronic device of claim 17 , wherein the action comprises at least one of: waking up the electronic device, performing automatic exposure, performing color balance, performing focus control, and identifying contextual information.