IP Library Granted Patent US 11,831,981
Granted Patent B2
US 11,831,981 · App. 17/400,502 · Granted Nov 28, 2023

Sensor system architecture with feedback loop and multiple power states

Inventor: Walter Nistico (Redwood City, CA)
Assignee: Apple Inc.
H04N23/651G06T1/20G06T7/10H04N23/667
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,831,981
App. No.
17/400,502
Granted
Nov 28, 2023
Kind
B2
Abstract

In one implementation, a system includes an event sensor with a pixel array and an image pipeline. The pixel array is configured to operate a first subset of pixels in an active state and a second subset of pixels in an inactive state. The event sensor is configured to output pixel events. Each respective pixel event is generated in response to a specific pixel within the first subset of pixels detecting a change in light intensity that exceeds a comparator threshold. The image pipeline is configured to consume image data derived from the pixel events and communicate feedback information to the event sensor based on the image data. The feedback information causes a pixel within the first subset of pixels to transition from the active state to another state.

Claims (37)

1. A system comprising:

an event sensor with a pixel array configured to operate a first subset of pixels in an active state and a second subset of pixels in an inactive state, the event sensor configured to output pixel events, each respective pixel event generated in response to a specific pixel within the first subset of pixels detecting a change in light intensity that exceeds a comparator threshold; and

an image pipeline configured to consume image data derived from the pixel events and communicate feedback information to the event sensor based on the image data, the feedback information causing a pixel within the first subset of pixels to transition from the active state to another state.

2. The system of claim 1 , wherein the feedback information corresponds to a region of interest that is tracked by the image pipeline within the image data.

3. The system of any of claim 1 , wherein each pixel in the second subset of pixels operates in a shutdown power mode.

4. The system of any of claim 1 , wherein the pixel array further includes a third subset of pixels in a standby state, each pixel in the third subset of pixels operating in a reduced power mode.

5. The system of claim 4 , wherein the feedback information causes the pixel within the first subset of pixels to transition to the standby state.

6. A system comprising:

an event sensor with a pixel array configured to have an active region of pixels operating in a full power mode and an inactive region of pixels operating in a shutdown power mode or a reduced power mode;

a processor; and

a computer-readable storage medium comprising instructions that upon execution by the processor cause the system to perform operations, the operations comprising:

outputting pixel events from the event sensor to an image pipeline, each respective pixel event generated in response to a specific pixel within the active region of pixels detecting a change in light intensity that exceeds a comparator threshold;

receiving feedback information from the image pipeline based on image data derived from the pixel events; and

directing a pixel within the active region of pixels to operate in the shutdown power mode or the reduced power mode in response to receiving the feedback information.

7. The system of claim 6 , wherein each pixel within the inactive region of pixels operates in the shutdown power mode, and wherein the pixel array further includes a standby region of pixels operating in the reduced power mode.

8. The system of claim 7 , wherein the standby region of pixels intervenes between the active region of pixels and the inactive region of pixels within the pixel array.

9. The system of any of claim 6 , wherein a subset of pixels within the active region of pixels operate in the shutdown power mode or the reduced power mode.

10. The system of any of claim 6 , wherein outputting the pixel events comprises: outputting bins of pixel events from the event sensor to the image pipeline.

11. The system of claim 10 , wherein the bins of pixel events are output from the event sensor on a periodic basis.

12. The system of claim 11 , wherein the periodic basis is synchronized with a global readout operation or a global reset operation of the event sensor.

13. The system of claim 10 , wherein the event sensor outputs each bin of pixel events after a predefined number of pixel events are generated.

14. The system of claim 10 , wherein each bin of pixel events is output as a two-dimensional tile of pixel events.

15. The system of claim 10 , wherein each bin of pixel events is output from the event sensor as a list of pixel events.

16. A system comprising:

a processor;

an image pipeline; and

a computer-readable storage medium comprising instructions that upon execution by the processor cause the system to perform operations, the operations comprising:

receiving, by the image pipeline, pixel events from an event sensor with a pixel array including a first subset of pixels in an active state and a second subset of pixels in an inactive state, each respective pixel event generated in response to a specific pixel within the first subset of pixels detecting a change in light intensity that exceeds a comparator threshold;

deriving image data from the pixel events with the image pipeline; and

generating feedback information with the image pipeline based on the image data, the feedback information causing the event sensor to direct a pixel within the first subset of pixels to transition from the active state to another state.

17. The system of claim 16 , wherein generating the feedback information comprises:

tracking a region of interest within the image data with the image pipeline.

18. The system of any of claim 16 , further comprising:

an optical source configured to emit light towards a scene disposed within a field of view of the event sensor.

19. The system of claim 18 , wherein the instructions, when executed, further cause the system to perform additional operations, the additional operations comprising:

pulsing the optical source at a defined frequency to cause pixels within the first subset of pixels to generate event data at a rate that is proportional to the defined frequency.

20. The system of any of claim 16 , wherein the feedback information is an active region offset that defines a region of the pixel array corresponding to a region of interest that the image pipeline tracks within the image data.

Continuity (3)
Continuation PCTUS2020023750 · Mar 20, 2020
Provisional Application 62824493 · Mar 27, 2019
Related Publication 20210377453A1 · Dec 2, 2021
Cited By (1)
US 12,413,869