IP Library Granted Patent US 11,175,145
Granted Patent B2
US 11,175,145 · App. 16/239,009 · Granted Nov 16, 2021

System and method for precision localization and mapping

Inventors: Vaibhav Ghadiok (Palo Alto, CA); Michael Gleeson-May (Palo Alto, CA)
Assignee: Nauto, Inc.
G01C21/28G01C21/30G01C21/34G01C21/3476G05D1/0246G05D1/0274G06F16/2379G06F16/29G06F16/5866G06K9/00818G06K9/2054G06K9/46G06T7/73G06T7/74G05D2201/0213G06K9/6256G06K2209/21G06T2207/10016G06T2207/30204G06T2207/30241G06T2207/30244G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,175,145
App. No.
16/239,009
Granted
Nov 16, 2021
Kind
B2
Abstract

A method for localization and mapping, including recording an image at a camera mounted to a vehicle, the vehicle associated with a global system location; identifying a landmark depicted in the image with a landmark identification module of a computing system associated with the vehicle, the identified landmark having a landmark geographic location and a known parameter; extracting a set of landmark parameters from the image with a feature extraction module of the computing system; determining, at the computing system, a relative position between the vehicle and the landmark geographic location based on a comparison between the extracted set of landmark parameters and the known parameter; and updating, at the computing system, the global system location based on the relative position.

Claims (37)

1. An apparatus comprising:

a camera configured to be mounted to a vehicle for viewing a region forward of the vehicle during vehicle motion, the camera configured to provide an image, wherein the vehicle is associated with a global system location and a location error; and

a computing system associated with the vehicle, the computing system comprising:

a landmark identification module configured to identify a landmark depicted in the image, wherein the landmark is associated with a landmark geographic location and a known dimension, and

a feature extraction module configured to determine a landmark parameter based on the image,

wherein the computing system is configured to:

determine a relative position between the vehicle and the landmark geographic location based on the landmark parameter,

update the global system location based on the relative position,

set the location error to an error value after the global system location is updated, and

change the location error after the global system location is updated; and

wherein the computing system is configured to provide an output for assisting a control of the vehicle; and

wherein the landmark comprises a street sign having a set of corners, and wherein the feature extraction module is configured to determine a set of pixel positions in the image corresponding with the set of corners, and to determine the landmark parameter based on the set of pixel positions.

2. The apparatus of claim 1 , wherein the landmark is one of a plurality of landmarks depicted in the image, wherein the landmark identification module is configured to identify the plurality of landmarks depicted in the image, each of the plurality of landmarks associated with a corresponding landmark geographic location;

wherein the feature extraction module is configured to determine a landmark pose associated with each of the plurality of landmarks from the image; and

wherein the computing system is configured to determine a set of relative positions between the vehicle and each of the plurality of landmarks based on the determined landmark poses associated with the respective landmarks, and determine the global system location based on the set of relative positions.

3. The apparatus of claim 1 , wherein the computing system further comprises a communication module configured to:

transmit the global system location to a second vehicle associated with a second global system location;

determine a second relative position between the second vehicle and the vehicle; and

update the second global system location based on the global system location and the second relative position.

4. The apparatus of claim 1 , wherein the computing system is configured to determine another relative position between the vehicle and the landmark geographic location.

5. The apparatus of claim 1 , wherein the computing system is configured to retrieve the landmark geographic location of the landmark from a remote database.

6. The apparatus of claim 1 , wherein the computing system is configured to determine a unit region based on the global system location, and retrieve the landmark geographic location from a storage based on the unit region.

7. The apparatus of claim 1 , wherein the landmark parameter comprises a textual location indicator, and wherein the computing system is configured to determine a unit region based on the textual location indicator, and retrieve the landmark geographic location from a storage based on the unit region.

8. The apparatus of claim 1 , wherein the computing system is configured to determine a route based on an accumulation rate of the location error exceeding a threshold accumulation rate value.

9. The apparatus of claim 1 , wherein the vehicle comprises an autonomous vehicle.

10. The apparatus of claim 1 , wherein the computing system is configured to determine a speed value and a heading value associated with the vehicle based on the landmark parameter, and wherein the speed value is associated with a speed error, and wherein the heading value is associated with a heading error.

11. The apparatus of claim 10 , wherein the computing system is configured to repeatedly update the global system location based on the speed value and the heading value, and repeatedly update the location error based on the speed error and the heading error.

12. The apparatus of claim 1 , wherein the computing system is configured to obtain a wheel RPM value from a rotations-per-minute (RPM) sensor, and wherein the computing system is configured to determine a speed value based on the wheel RPM value in combination with a known wheel diameter.

13. The apparatus of claim 1 , wherein the landmark is one or a plurality of landmarks depicted in the image, wherein the landmark identification module is configured to identify the plurality of landmarks depicted in the image, wherein each of the plurality of landmarks is associated with a corresponding landmark geographic location;

wherein the feature extraction module is configured to determine a set of landmark parameters associated with each of the plurality of landmarks from the image; and

wherein the computing system is configured to determine a pattern based on the set of landmark parameters associated with the respective landmarks, and determine the global system location based on the pattern.

14. The apparatus of claim 1 , wherein the computing system is also configured to:

determine a temporal pattern based at least in part on the landmark parameter; and

determine a vehicle trajectory based on the temporal pattern.

15. The apparatus of claim 1 , wherein the apparatus is configured to record the image based on a satisfaction of a criterion.

16. The apparatus of claim 15 , wherein the criterion is associated with a threshold.

17. The apparatus of claim 1 , wherein the computing system is configured to update the global system location based on a predetermined frequency.

Assignments (4)
RELEASE OF SECURITY INTEREST Recorded Jun 19, 2026
From: ORIX GROWTH CAPITAL, LLC
To: NAUTO, INC.
Reel/Frame 075016/0824 →
SECURITY INTEREST Recorded Aug 8, 2025
From: NAUTO, INC.
To: ORIX GROWTH CAPITAL, LLC, AS AGENT
Reel/Frame 071976/0818 →
SECURITY INTEREST Recorded Nov 10, 2022
From: NAUTO, INC.
To: SILICON VALLEY BANK
Reel/Frame 061722/0392 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 3, 2019
From: GHADIOK, VAIBHAV; GLEESON-MAY, MICHAEL
To: NAUTO, INC.
Reel/Frame 047893/0288 →
Continuity (4)
Continuation 15673098 · Aug 9, 2017
Provisional Application 62492790 · May 1, 2017
Provisional Application 62372633 · Aug 9, 2016
Related Publication 20190137280A1 · May 9, 2019
Cited By (1)
US 12,591,887