IP Library Granted Patent US 10,024,965
Granted Patent B2
US 10,024,965 · App. 15/087,222 · Granted Jul 17, 2018

Generating 3-dimensional maps of a scene using passive and active measurements

Inventors: Youval Nehmadi (Nili, IL); Shmuel Ur (Shorashim, IL); Ronny Cohen (Ramat Hasharon, IL)
Assignee: VayaVision, Ltd.
G01S17/08G01C3/08G01C21/3602G01S17/023G01S17/42G01S17/89G01S17/936G05D1/0088G06K9/00201G06K9/00791G06K9/00805G06K9/00825G06K9/6202G06K9/6215G06K9/6267G06T7/20G06T15/00G05D2201/0213G06T2200/04G06T2207/10012G06T2207/10052G06T2207/30236G06T2207/30261Y10S901/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,024,965
App. No.
15/087,222
Granted
Jul 17, 2018
Kind
B2
Abstract

A method and apparatus for generating 3D-maps for acquiring three-dimensional (3D) maps are presented. The method includes analyzing at least one image acquired by a passive sensor to identify a plurality of objects in the at least one image; classifying the plurality of objects; determining, based on the classification, whether to passively measure a distance to each of the plurality of objects; passively measuring the distance to at least one of the plurality of objects based on the determination; actively measuring a distance to some of the plurality of objects, wherein the distance to one of the same of the plurality of objects is actively measured when the distance to the object cannot be passively measured; and generating a 3D map of a scene based on the distance measured to each of the plurality of objects.

Claims (63)

1. A method for acquiring three-dimensional (3D) maps, comprising:

analyzing at least one image acquired by a passive sensor to identify a plurality of objects in the at least one image;

classifying the plurality of objects as any of: a stationary object, a non-stationary object, a suspicious object, and a non-suspicious object;

determining, based on the classification, whether to passively measure a distance to each of the plurality of objects;

passively measuring the distance to at least one of the plurality of objects based on the determination;

wherein the distance to the at least one object is passively measured, for each object classified as a stationary object, when a previous distance to the object is not available;

actively measuring a distance to some of the plurality of objects, wherein the distance to one of the same of the plurality of objects is actively measured when the distance to the object cannot be passively measured; and

generating a 3D map of a scene based on the distance measured to each of the plurality of objects.

2. The method of claim 1 , wherein analyzing the at least one image further comprises:

segmenting the at least one image to render a segmentation map; and

identifying the plurality of objects in the segmentation map.

3. The method of claim 1 , further comprising:

identifying a type of each of the plurality of objects; and

classifying each of the plurality of objects based on the identified type of the object.

4. The method of claim 1 , further comprising:

comparing the at least one image to a previously acquired image; and

identifying a change in a location of each of the plurality of objects based on the comparison, wherein each of the plurality of objects is classified based on the change in the location of the object.

5. The method of claim 1 , further comprising:

determining, for each object of the plurality of objects, whether the object is identified in at least one geographic map; and

classifying each object identified in the at least one geographic map as a stationary object.

6. The method of claim 1 , wherein the passively measuring the distance to the at least one object further comprising:

determining if the passive sensor is in motion;

determining the distance to the least one object using a motion sensor, when the passive sensor is in motion; and

determining the distance to the least one object based on the number of the pixels to the object and a known reference measurement, when the passive sensor is not in motion.

7. The method of claim 6 , wherein the known reference measurement is at least one of: a number of pixels from a valid reference object, and a dimension of the object.

8. The method of claim 1 , wherein passively measuring the distance further comprising, for each object classified as a non-stationary object and as a non-suspicious object:

determining a boundary speed of the object;

determining a direction of movement of the object; and

estimating the distance to the object based on the determined direction and the determined boundary speed.

9. The method of claim 8 , further comprising:

determining if the estimated distance is below a safe distance threshold; and

actively measuring the distance to the at least one object, when the estimated distance is below the safe distance threshold.

10. The method of claim 1 , wherein the distance is actively measured to any object classified as any of: a non-stationary object, and a suspicious object.

11. The method of claim 1 , wherein the distance to one of the plurality of objects is actively measured when a reliable passive measurement for the object cannot be determined.

12. The method of claim 1 , wherein actively measuring the distance to some of the plurality of objects further comprises:

performing a number of laser measurements for each actively measured object, wherein the number of laser measurements is determined based on a required resolution.

13. The method of claim 12 , further comprising:

computing, for each actively measured object, a distance to each point in the object based on the laser measurements.

14. The method of claim 1 , wherein generating the 3D-map further comprising:

rendering a 3D-surface of each object using the distance measurements.

15. The method of claim 1 , wherein the scene is a field of view captured by an apparatus for causing generation of the 3D-map.

16. A non-transitory computer-readable medium having stored thereon instructions for executing the method according to claim 1 .

17. An apparatus for acquiring three-dimensional (3D) maps, comprising:

a passive sensor for acquiring images of a scene;

an active sensor for measuring distances from objects in the scene; and

a processing system, wherein the processing system is configured to:

analyze at least one image acquired by the passive sensor to identify a plurality of objects in the at least one image;

classify the plurality of objects as any of: a stationary object, a non-stationary object, a suspicious object, and a non-suspicious object;

determine, based on the classification, whether to passively measure a distance to each of the plurality of objects;

passively measure the distance to at least one of the plurality of objects based on the determination;

wherein the distance to the at least one object is passively measured, for each object classified as a stationary object, when a previous distance to the object is not available;

actively measure, by the active sensor, a distance to some of the plurality of objects, wherein the distance to one of the plurality of objects is actively measured when the distance to the object cannot be passively measured; and

generate a 3D map of a scene based on the distance measured to each of the plurality of objects.

18. A method for acquiring three-dimensional (3D) maps, comprising:

acquiring at least one image by a passive sensor;

segmenting the at least one image to render a segmentation map;

classifying a plurality of pixels in each segment identified in the segmentation map as any of: a stationary pixel, a non-stationary pixel, a suspicious pixel, and a non-suspicious pixel;

determining, based on the classification, whether to passively measure a distance to each of the plurality of pixels;

passively measuring the distance to at least one of the plurality of pixels based on the determination;

wherein the distance to the at least one pixel is passively measured, for each pixel classified as a stationary pixel, when a previous distance to the pixel is not available;

actively measuring a distance value to each pixel of a first number of pixels in each segment identified in the segmentation map, wherein the first number of pixels is less than the number of pixels in each segment;

computing a distance value to each pixel in the segment using the distance of pixel in the first number of pixels; and

generating a 3D map of a scene based on the measured and computed distance values.

Assignments (6)
CORRECTION BY DECLARATION ERRONEOUSLY FILED AGAINST REEL 063965 FRAME 0233 Recorded Mar 8, 2024
From: VAYAVISION SENSING, LTD.
To: VAYAVISION SENSING, LTD.
Reel/Frame 066762/0712 →
CORRECTION BY DECLARATION ERRONEOUSLY FILED AGAINST REEL 063316 FRAME 0716 Recorded Mar 8, 2024
From: VAYAVISION SENSING, LTD.
To: VAYAVISION SENSING, LTD.
Reel/Frame 066771/0863 →
SECURITY INTEREST Recorded Jun 15, 2023
From: LEDDARTECH INC.
To: TSX TRUST COMPANY
Reel/Frame 063965/0233 →
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY NAME PREVIOUSLY RECORDED AT REEL: 038163 FRAME: 0032. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jun 2, 2023
From: NEHMADI, YOUVAL; UR, SHMUEL; COHEN, RONNY
To: VAYAVISION SENSING, LTD.
Reel/Frame 063823/0363 →
SECURITY INTEREST Recorded Apr 13, 2023
From: LEDDARTECH INC.
To: FÉDÉRATION DES CAISSES DESJARDINS DU QUÉBEC
Reel/Frame 063316/0716 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 31, 2016
From: NEHMADI, YOUVAL; UR, SHMUEL; COHEN, RONNY
To: VAYAVISION, LTD.
Reel/Frame 038163/0032 →
Continuity (3)
Provisional Application 62141296 · Apr 1, 2015
Provisional Application 62160988 · May 13, 2015
Related Publication 20160292905A1 · Oct 6, 2016
Cited By (2)
US 12,352,591 US 12,560,719