IP Library Granted Patent US 9,939,813
Granted Patent B2
US 9,939,813 · App. 15/273,190 · Granted Apr 10, 2018

Systems and methods for refining landmark positions

Inventors: Amnon Shashua (Jerusalem, IL); Yoram Gdalyahu (Jerusalem, IL); Daniel Braunstein (Jerusalem, IL)
Assignee: Mobileye Vision Technologies Ltd.
G05D1/0088B60W30/14B60W30/18B62D15/025G01C21/14G01C21/165G01C21/32G01C21/34G01C21/3407G01C21/3476G01C21/36G01C21/3623G01C21/3644G01C21/3691G05D1/0212G05D1/0221G05D1/0246G05D1/0251G05D1/0253G05D1/0278G05D1/0287G06F17/30241G06F17/30377G06K9/00791G06K9/00798G06K9/00818G08G1/0112G08G1/09623G08G1/096725G08G1/096805G08G1/167B60W2420/42B60W2550/22B60W2710/18B60W2710/20B60W2720/10G01S19/10G05D2201/0213G06T7/00G06T2207/20081G06T2207/30256G06T2207/30261H04L67/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,939,813
App. No.
15/273,190
Granted
Apr 10, 2018
Kind
B2
Abstract

A system is provided for determining a location of a landmark for use in navigation of an autonomous vehicle. The system includes a processor programmed to receive a measured position of the landmark. The processor is also programmed to determine a refined position of the landmark based on the measured position of the landmark and at least one previously acquired position for the landmark. The measured position and the previously acquired position are determined based on acquisition of an environmental image associated with the host vehicle, analysis of the environmental image to identify the landmark, reception of global positioning system (GPS) data representing a location of the host vehicle, analysis of the environmental image to determine a relative position of the identified landmark with respect to the host vehicle, and determination of a globally localized position of the landmark based on the GPS data and the relative position.

Claims (54)

1. A system for determining a location of a landmark for use in navigation of an autonomous vehicle, the system comprising:

at least one processor programmed to:

receive a measured position of the landmark;

determine a refined position of the landmark based on the measured position of the landmark and at least one previously acquired position for the landmark, wherein the measured position and the at least one previously acquired position are determined based on acquisition, from a camera associated with a host vehicle, of at least one environmental image associated with the host vehicle, analysis of the at least one environmental image to identify the landmark in the environment of the host vehicle, reception of global positioning system (GPS) data representing a location of the host vehicle, analysis of the at least one environmental image to determine a relative position of the identified landmark with respect to the host vehicle, and determination of a globally localized position of the landmark based on at least the GPS data and the determined relative position;

selectively preclude transmission of the refined position of the landmark or the measured position of the landmark to a server, the server storing a sparse data model including landmark position information, based on at least one value indicative of a determined confidence level; and

selectively preclude transmission of a distance between two measured landmark positions to the server based on the at least one value indicative of the determined confidence level.

2. The system of claim 1 , wherein the landmark includes at least one of a traffic sign, an arrow, a lane marking, a dashed lane marking, a traffic light, a stop line, a directional sign, a landmark beacon, or a lamppost.

3. The system of claim 1 , wherein analysis of the at least one image to determine the relative position of the identified landmark with respect to the vehicle includes calculating a distance based on a scale associated with the at least one image.

4. The system of claim 1 , wherein analyzing the at least one image to determine the relative position of the identified landmark with respect to the vehicle includes calculating a distance based on an optical flow associated with at least two images.

5. The system of claim 1 , wherein the GPS data is received from a GPS device included in the host vehicle.

6. The system of claim 1 , wherein the camera is included in the host vehicle.

7. The system of claim 1 , wherein determining the refined position of the landmark includes averaging the measured position of the landmark with the at least one previously acquired position.

8. The system of claim 1 , wherein the landmark is associated with a road segment, and the at least one value indicative of the determined confidence level is associated with the road segment.

9. The system of claim 1 , wherein the landmark is associated with a geographical region, and the at least one value indicative of the determined confidence level is associated with the geographical region.

10. The system of claim 1 , wherein the landmark is associated with a local map area, and the at least one value indicative of the determined confidence level is associated with the local map area.

11. The system of claim 1 , wherein the at least one processor is further programmed to receive the at least one value indicative of the determined confidence level from the server.

12. The system of claim 1 , wherein the server is configured to selectively control data flow from the autonomous vehicle based on the at least one value indicative of the determined confidence level.

13. The system of claim 1 , wherein the sparse data model has a landmark density of no more than twenty landmarks per kilometer.

14. The system of claim 1 , wherein the server is configured to receive additional information from the autonomous vehicle other than the refined position of the landmark and the measured position of the landmark.

15. The system of claim 14 , wherein the additional information includes an image captured by an image capturing device.

16. The system of claim 1 , wherein the at least one processor is included in the autonomous vehicle.

17. The system of claim 1 , wherein the at least one processor is included in the server.

18. A method for determining a location of a landmark for use in navigation of an autonomous vehicle, the method comprising:

receiving a measured position of the landmark;

determining a refined position of the landmark based on the measured position of the landmark and at least one previously acquired position for the landmark, wherein the measured position and the at least one previously acquired position are determined based on:

acquisition, from a camera associated with a host vehicle, of at least one environmental image associated with the host vehicle,

analysis of the at least one environmental image to identify the landmark in the environment of the host vehicle,

reception of global positioning system (GPS) data representing a location of the host vehicle,

analysis of the at least one environmental image to determine a relative position of the identified landmark with respect to the host vehicle, and

determination of a globally localized position of the landmark based on at least the GPS data and the determined relative position;

selectively precluding transmission of the refined position of the landmark or the measured position of the landmark to a server, the server storing a sparse data model including landmark position information, based on at least one value indicative of a determined confidence level; and

selectively preclude transmission of a distance between two measured landmark positions to the server based on the at least one value indicative of the determined confidence level.

19. The method of claim 18 , wherein the landmark includes at least one of a traffic sign, an arrow, a lane marking, a dashed lane marking, a traffic light, a stop line, a directional sign, a landmark beacon, or a lamppost.

20. The method of claim 18 , wherein analysis of the at least one image to determine the relative position of the identified landmark with respect to the vehicle includes calculating a distance based on a scale associated with the at least one image.

21. The method of claim 18 , wherein analysis of the at least one image to determine the relative position of the identified landmark with respect to the vehicle includes calculating a distance based on an optical flow associated with at least two images.

22. The method of claim 18 , wherein the GPS data is received from a GPS device included in the host vehicle.

23. The method of claim 18 , wherein the camera is included in the host vehicle.

24. The method of claim 18 , wherein determining the refined position of the landmark includes averaging the measured position of the landmark with the at least one previously acquired position.

25. The method of claim 18 , wherein the method is performed by at least one processor included in the autonomous vehicle.

26. The method of claim 18 , wherein the method is performed by at least one processor included in the server.

27. An autonomous vehicle, comprising:

a body; and

at least one processor programmed to:

receive a measured position of a landmark;

determine a refined position of the landmark based on the measured position of the landmark and at least one previously acquired position for the landmark, wherein the at least one processor is further programmed to determine the measured position and the at least one previously acquired position based on acquisition, from a camera associated with the vehicle, of at least one environmental image associated with the vehicle, analysis of the at least one environmental image to identify the landmark in the environment of the vehicle, reception of global positioning system (GPS) data representing a location of the vehicle, analysis of the at least one environmental image to determine a relative position of the identified landmark with respect to the vehicle, and determination of a globally localized position of the landmark based on at least the GPS data and the determined relative position;

selectively preclude transmission of the refined position of the landmark to a server, the server storing a sparse data model including landmark position information, based on at least one value indicative of a determined confidence level; and

selectively preclude transmission of a distance between two measured landmark positions to the server based on the at least one value indicative of the determined confidence level.

28. The vehicle of claim 27 , wherein the landmark includes at least one of a traffic sign, an arrow, a lane marking, a dashed lane marking, a traffic light, a stop line, a directional sign, a landmark beacon, or a lamppost.

29. The vehicle of claim 27 , wherein analysis of the at least one image to determine the relative position of the identified landmark with respect to the vehicle includes calculating a distance based on a scale associated with the at least one image.

30. The vehicle of claim 27 , wherein analyzing the at least one image to determine the relative position of the identified landmark with respect to the vehicle includes calculating a distance based on an optical flow associated with at least two images.

31. The vehicle of claim 27 , wherein the GPS data is received from a GPS device included in the host vehicle.

32. The vehicle of claim 27 , wherein determining the refined position of the landmark includes averaging the measured position of the landmark with the at least one previously acquired position.

33. The vehicle of claim 27 , wherein the at least one processor is further programmed to receive the at least one value indicative of the determined confidence level from the server.

34. The vehicle of claim 27 , wherein the sparse data model has a landmark density of no more than twenty landmarks per kilometer.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2017
From: BRAUNSTEIN, DANIEL; GDALYAHU, YORAM; SHASHUA, AMNON
To: MOBILEYE VISION TECHNOLOGIES LTD.
Reel/Frame 041716/0944 →
Continuity (22)
Continuation PCTUS2016017411 · Feb 10, 2016
Provisional Application 62114091 · Feb 10, 2015
Provisional Application 62164055 · May 20, 2015
Provisional Application 62170728 · Jun 4, 2015
Provisional Application 62181784 · Jun 19, 2015
Provisional Application 62192576 · Jul 15, 2015
Provisional Application 62215764 · Sep 9, 2015
Provisional Application 62219733 · Sep 17, 2015
Provisional Application 62261578 · Dec 1, 2015
Provisional Application 62261598 · Dec 1, 2015
Provisional Application 62267643 · Dec 15, 2015
Provisional Application 62269818 · Dec 18, 2015
Provisional Application 62270408 · Dec 21, 2015
Provisional Application 62270418 · Dec 21, 2015
Provisional Application 62270431 · Dec 21, 2015
Provisional Application 62271103 · Dec 22, 2015
Provisional Application 62274883 · Jan 5, 2016
Provisional Application 62274968 · Jan 5, 2016
Provisional Application 62275007 · Jan 5, 2016
Provisional Application 62275046 · Jan 5, 2016
Provisional Application 62277068 · Jan 11, 2016
Related Publication 20170010121A1 · Jan 12, 2017