IP Library Granted Patent US 9,628,777
Granted Patent B2
US 9,628,777 · App. 14/363,681 · Granted Apr 18, 2017

Method of 3D reconstruction of a scene calling upon asynchronous sensors

Inventors: Ryad Benosman (Paris, FR); Joao Carneiro (Paris, FR); Sio-Hoi Ieng (Paris, FR)
Assignees: UNIVERSITE PIERRE ET MARIE CURIE (PARIS 6); CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE
H04N13/0239H04N13/0242H04N13/0296
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,628,777
App. No.
14/363,681
Granted
Apr 18, 2017
Kind
B2
Abstract

A method of 3D reconstruction of a scene, including the implementation of at least two sensors each having a series of elementary receivers arranged so as to view the scene according to distinct respective solid angles and which are each sensitive to a physical characteristic of a signal received by the receiver originating from the scene. The sensors are adapted for emitting an asynchronous stream of events indicative of a modification of the signal received by one of the elementary receivers at a given instant, as well as the implementation of a 3D reconstruction algorithm having a step of pairing elementary receivers of each of the sensors utilizing signals generated by the sensors, the pairing including pairing between them events generated by each of the sensors and temporarily close together.

Claims (21)

1. A method of 3D reconstruction of a scene, including:

the implementation of at least two sensors (C 1 ,C 2 ) each including a series of elementary receivers arranged to observe the scene from separate respective solid angles and which are each sensitive to a physical characteristic of a signal received by said receiver originating from the scene;

said sensors being each adapted to emit an asynchronous stream of events (E i ) that are each indicative of a modification of the signal received by one of the elementary receivers at a given instant;

the implementation of a 3D reconstruction algorithm including:

matching elementary receivers of each of the sensors employing signals generated by the sensors, said matching including matching events generated by receivers of each sensors that relates to one and the same pattern of the scene;

determining by triangulation the position of the point of the scene corresponding to that pattern of the scene that has been seen by the two receivers which generated the selected events, for integrating it in to the 3D reconstruction of the scene;

characterized in that the matching step includes:

performing a preliminary selection of temporally close events by retaining, for one given source event generated by one of the sensors, a set of events generated by the other sensor which have occurred in a given time window (ΔT) around the instant of occurrence of the source event;

if the preliminary selection retains several events generated by the other sensor, filtering these events and retaining only the one that corresponds to the source event, by applying a sifting procedure.

2. The method as claimed in claim 1 , wherein each event comprises the following data:

data making it possible to identify the elementary receiver having generated the event;

an indicator of the modification of the signal perceived by said elementary receiver and taking at least a first value if the signal received by the elementary receiver has increased, and a second value if it has decreased;

the instant of occurrence of the modification of the signal.

3. The method as claimed in claim 2 , wherein the sensors have a focal plane, and wherein the sifting procedure comprises:

for each selected event, computing a distance between a point of the focal plane of the second sensor corresponding to the elementary receiver having generated said selected event and an epipolar line in said focal plane associated with the elementary receiver of the first sensor having generated the source event, and

eliminating all the selected events for which the distance thus determined is above a given limit D.

4. The method as claimed in claim 2 , wherein the sifting procedure comprises eliminating all the selected events the indicator of which does not have the same value as the source event.

5. The method as claimed in claim 2 , wherein the sifting procedure comprises eliminating all the selected events having already been the subject of a matching.

6. The method as claimed in claim 1 , wherein at least three asynchronous sensors (C 1 ,C 2 ,C 3 ) are implemented then, after temporal selection, an event generated by each of the sensors is retained in such a way that each of said retained events is found at the intersection of the epipolar lines generated by the other retained events in the corresponding focal plane.

7. The method as claimed in claim 1 , wherein the sensors comprise cameras of CCD or CMOS type combined with post-processing adapted to generate the asynchronous stream.

8. The method as claimed in claim 1 , wherein the sensors are of the type including receivers which, in an independent and continuous way, quantify relative changes in luminous intensity to generate events, in such a way as to deliver an asynchronous stream of events.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE'S ADDRESS PREVIOUSLY RECORDED ON REEL 045953 FRAME 0583. ASSIGNOR(S) HEREBY CONFIRMS THE MERGER. Recorded Jun 21, 2018
From: UNIVERSITE PIERRE ET MARIE CURIE (PARIS 6)
To: SORBONNE UNIVERSITE
Reel/Frame 047331/0847 →
MERGER Recorded May 31, 2018
From: UNIVERSITE PIERRE ET MARIE CURIE (PARIS 6)
To: SORBONNE UNIVERSITE
Reel/Frame 045953/0583 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 9, 2014
From: BENOSMAN, RYAD; CARNEIRO, JOAO; IENG, SIO-HOI
To: UNIVERSITE PIERRE ET MARIE CURIE (PARIS 6); CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE
Reel/Frame 033055/0941 →
Priority Claims (1)
FR 11 61320 · Dec 8, 2011 · national
Continuity (1)
Related Publication 20140333730A1 · Nov 13, 2014