IP Library › Granted Patent US 12,062,217
Granted Patent B2
US 12,062,217 · App. 17/413,347 · Granted Aug 13, 2024

Method of tracking objects in a scene

Inventors: Amos Sironi (Questembert, FR); Pierre De Tournemire (Paris, FR); Daniele Perrone (Noisy le Grand, FR)
Assignee: PROPHESEE
G06V10/255G06V20/56
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,062,217
App. No.
17/413,347
Granted
Aug 13, 2024
Kind
B2
Abstract

The present invention relates to a method of tracking objects in a scene observed by an event-based sensor that produces events asynchronously from a matrix of sensing elements, wherein, from each sensing element of the matrix, respective events are produced as a function of variations of light incident on the sensing element, the method comprising: detecting, by a first process, an object in the scene from asynchronous events produced by the event-based sensor; and determining, by a second process, a track associated to the detected object from asynchronous events produced by the event-based sensor; wherein the first process has a time resolution lower than the second process.

Claims (46)

1. A method of tracking objects in a scene observed by an event-based sensor having a matrix of sensing elements that produce events asynchronously,

wherein, from each sensing element of the matrix, respective events are produced as a function of variations of light incident on the sensing element,

the method comprising:

detecting, by a first process, an object in the scene from asynchronous events produced by the event-based sensor; and

determining, by a second process, a track associated to the detected object from asynchronous events produced by the event-based sensor,

wherein the first process has a time resolution lower than the second process.

2. The method of claim 1 , wherein the first process comprises determining a category of the detected object.

3. The method of claim 1 , wherein a new iteration of the first process is run according to a predefined time sequence.

4. The method of claim 1 , wherein the first process comprises determining a region of interest in the matrix of sensing elements, the region of interest including the detected object; and

wherein the second process comprises determining the track based on asynchronous events produced by sensing elements in a region including the region of interest.

5. The method of claim 4 , wherein the second process comprises updating the region of interest based on asynchronous events produced in a region including the region of interest.

6. The method of claim 4 , wherein the second process further comprises running a new iteration of the first process based on a parameter of the region of interest, wherein the parameter is one of a size of the region of interest and a shape of the region of interest.

7. The method of claim 4 , wherein the second process further comprises running a new iteration of the first process if a number of events produced in the region of interest during a time interval exceeds a threshold.

8. The method of claim 1 , further comprising:

determining associations between detected objects and determined tracks; and

updating a set of detected objects and/or a set of determined tracks based on the associations.

9. The method of claim 8 , further comprising:

if a given object among detected objects is associated with a given track among determined tracks, updating the given track based on a position of the given object.

10. The method of claim 8 , further comprising:

if a given object among detected objects is not associated with any track among determined tracks, initializing a new track corresponding to the given object;

if a given track among determined tracks is not associated with any object among detected objects, deleting the given track;

if a first object and a second object among detected objects are both associated with a first track among determined tracks, initializing a second track from the first track, each of the first and second tracks corresponding to a respective object among the first and second objects; and

if a given object among detected objects is associated to a first track and a second track among determined tracks, merging the first track and the second track.

11. The method of claim 1 , wherein the second process further comprises:

determining, for each detected object, a respective set of regions in the matrix of sensing elements, each region of the set of regions being associated to a respective time;

receiving events produced by the event-based sensor during a predefined time interval;

determining groups of events, each group comprising at least one event among said received events;

for each group of events, determining a respective zone in the matrix of sensing elements, the zone including all events of the group of events;

determining associations between the determined zones and the determined regions; and

updating a set of detected objects and/or a set of determined tracks based on the associations.

12. The method of claim 1 , wherein the first process further comprises determining a segmentation mask for the detected object, and wherein the second process comprises updating the segmentation mask based on asynchronous events produced in a neighborhood of the segmentation mask.

13. The method of claim 11 , further comprising:

extracting first descriptive parameters associated to events received from the event-based sensor;

extracting second descriptive parameters associated to events corresponding to the segmentation mask; and

updating the segmentation mask by comparing a distance between one of the first descriptive parameters and one of the second descriptive parameters to a predefined value,

wherein the first and the second descriptive parameters associated to an event comprise at least one of a polarity of the event and a speed of the event.

14. A signal processing unit, comprising:

an interface for connecting to an event-based sensor having a matrix of sensing elements that produce events asynchronously from light received from a scene, wherein, from each sensing element of the matrix, respective events are produced as a function of variations of light incident on the sensing element; and

a processor configured to:

detect, by a first process, an object in the scene from asynchronous events produced by the event-based sensor; and

determine, by a second process, a track associated to the detected object from asynchronous events produced by the event-based sensor;

wherein the first process has a time resolution lower than the second process.

15. A computer-readable medium having computer code stored thereon for execution in a processor associated with an event-based sensor having a matrix of sensing elements that produce events asynchronously from light received from a scene, wherein, from each sensing element of the matrix, respective events are produced as a function of variations of light incident on the sensing element, wherein execution of the instructions causes the processor to:

detect, by a first process, an object in the scene from asynchronous events produced by the event-based sensor; and

determine, by a second process, a track associated to the detected object from asynchronous events produced by the event-based sensor,

wherein the first process has a time resolution lower than the second process.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 29, 2022
From: SIRONI, AMOS; DE TOURNEMIRE, PIERRE; PERRONE, DANIELE
To: PROPHESEE
Reel/Frame 062239/0722 →
Priority Claims (1)
EP 18212488 · Dec 13, 2018 · regional
Continuity (1)
Related Publication 20220036110A1 · Feb 3, 2022