IP Library Granted Patent US 11,212,468
Granted Patent B2
US 11,212,468 · App. 16/297,273 · Granted Dec 28, 2021

Event-based vision sensor with direct memory control

Inventors: Raphael Berner (Plan-les-Ouates, CH); Christian Brändli (Baden, CH); Massimo Zannoni (Zürich, CH)
Assignee: Sony Advanced Visual Sensing AG
H04N5/351H04N5/345H04N5/378H04N5/379H04N5/3745H04N5/3765H04N5/37452
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,212,468
App. No.
16/297,273
Granted
Dec 28, 2021
Kind
B2
Abstract

An event-based vision sensor system comprises an event-based pixel array including pixels that detect light, a readout circuit for reading out events associated with light received by each pixel and with the position it occupies in the array, and a memory controller that enables writing of each event, or event-related data, associated with a certain pixel to a certain address in a memory.

Claims (35)

1. An event-based vision sensor system, comprising:

an event-based pixel array comprising pixels that detect light;

a readout circuit for reading out events associated with light received by each pixel and with the position it occupies in the array; and

a memory controller that enables writing of each event, or event-related data, associated with a certain pixel to a certain address in a memory, the events relating to whether On events or Off events occurred in each pixel;

wherein the memory controller is on the same chip as the event-based pixel array and adds the addresses to an index buffer and a processing unit reads the index buffer and updates a read pointer in the index buffer to a last memory address that has been read.

2. The system as claimed in claim 1 , wherein the memory controller that ensures that the address in the memory in which that event or event-related data is written is a function of the position in the pixel array occupied by the pixel that generated that event.

3. The system as claimed in claim 1 , wherein the memory can be written by the memory controller and independently read from outside.

4. The system as claimed in claim 1 , which is implemented as three-dimensional integrated circuit.

5. The system as claimed in claim 1 , wherein the memory controller implements a one-to-one mapping between pixels of the event-based pixel array and addresses in the memory.

6. The system as claimed in claim 1 , wherein the memory is on the same chip as the event-based pixel array.

7. The system as claimed in claim 1 , wherein the memory controller stores timestamps in the memory.

8. The system as claimed in claim 1 , wherein the memory controller maintains a buffer index in the memory for last time stamp map also stored in the memory.

9. The system as claimed in claim 1 , wherein the memory controller sends timestamps followed by addresses of pixels that detected a change to the memory.

10. The system as claimed in claim 1 , wherein the memory controller includes a ring buffer in which packages of events received from the readout circuit are stored to successive addresses that wrap to the smallest when a maximum index is reached.

11. The system as claimed in claim 1 , wherein the processing unit accesses the memory and performs elaboration on the events stored in the memory.

12. The system as claimed in claim 11 , wherein the elaboration performed by the processing unit includes estimating a speed of an object from the events.

13. The system as claimed in claim 1 , wherein the memory controller waits for events and then receives the events from the readout circuit, the memory controller assigns timestamps read from a clock source to each of the events received from the readout circuit RO, and calculates addresses of the memory to which the timestamps for the events are written according to a one-to-one mapping between pixels of the event-based pixel array and addresses in the memory.

14. A method of operation of an event-based vision sensor system, comprising:

detecting light with an event-based pixel array;

reading out events associated with light received by each pixel and with the position it occupies in the array with a readout circuit; and

a memory controller writing of each event, or event-related data, associated with each pixel to different address in a memory;

wherein the memory controller is on the same chip as the event-based pixel array and adds the addresses to an index buffer and a processing unit reads the index buffer and updates a read pointer in the index buffer to a last memory address that has been read.

15. The method as claimed in claim 10 , further comprising the memory controller ensuring that the position of that address in the memory in which that event or event-related data is written, is a function of the position in the pixel array occupied by the pixel that generated that event.

16. The method as claimed in claim 10 , wherein the memory can be written by the memory controller and independently read from outside.

17. The method as claimed in claim 10 , which is implemented as three-dimensional integrated circuit.

18. The method as claimed in claim 14 , further comprising the memory controller mapping between pixels of the event-based pixel array and addresses in the memory on a one-to-one basis.

19. The method as claimed in claim 14 , wherein the memory is on the same chip as the event-based pixel array.

20. The method as claimed in claim 14 , further comprising the memory controller storing timestamps in the memory.

21. The method as claimed in claim 14 , further comprising the memory controller maintaining a buffer index in the memory for last time stamp map also stored in the memory.

22. The method as claimed in claim 14 , further comprising the memory controller sending timestamps followed by addresses of pixels that detected a change to the memory.

23. An event-based vision sensor system, comprising:

an event-based pixel array comprising pixels that detect light;

a readout circuit for reading out events associated with light received by each pixel and with the position it occupies in the array; and

a memory controller that enables writing of each event, or event-related data, associated with a certain pixel to a certain address in a memory with a one-to-one mapping between pixels of the event-based pixel array and addresses in the memory;

wherein the memory controller is on the same chip as the event-based pixel array and the memory can be written by the memory controller and independently read from outside, and the memory controller adds the addresses to an index buffer and a processing unit reads the index buffer and updates a read pointer in the index buffer to a last memory address that has been read.

Assignments (2)
CHANGE OF NAME Recorded Aug 21, 2020
From: INSIGHTNESS AG
To: SONY ADVANCED VISUAL SENSING AG
Reel/Frame 053568/0869 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2019
From: BERNER, RAPHAEL; BRÄNDLI, CHRISTIAN; ZANNONI, MASSIMO
To: INSIGHTNESS AG
Reel/Frame 048739/0887 →
Continuity (2)
Provisional Application 62642842 · Mar 14, 2018
Related Publication 20190289230A1 · Sep 19, 2019