IP Library Granted Patent US 10,209,081
Granted Patent B2
US 10,209,081 · App. 15/673,098 · Granted Feb 19, 2019

System and method for precision localization and mapping

Inventors: Vaibhav Ghadiok (Palo Alto, CA); Michael Gleeson-May (Palo Alto, CA)
Assignee: Nauto, Inc.
G01C21/28G01C21/34G06F17/30241G06F17/30268G06F17/30377G06K9/00818G06K9/2054G06K9/46G06T7/74G05D1/0246G06K9/6256G06K2209/21G06T2207/30204G06T2207/30241G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,209,081
App. No.
15/673,098
Granted
Feb 19, 2019
Kind
B2
Abstract

A method for localization and mapping, including recording an image at a camera mounted to a vehicle, the vehicle associated with a global system location; identifying a landmark depicted in the image with a landmark identification module of a computing system associated with the vehicle, the identified landmark having a landmark geographic location and a known parameter; extracting a set of landmark parameters from the image with a feature extraction module of the computing system; determining, at the computing system, a relative position between the vehicle and the landmark geographic location based on a comparison between the extracted set of landmark parameters and the known parameter; and updating, at the computing system, the global system location based on the relative position.

Claims (25)

1. A method for vehicular localization and mapping, comprising:

at a camera mounted to a vehicle and oriented to define a field of view including a region forward of the vehicle during vehicle motion, wherein the vehicle is associated with a global system location, wherein the global system location is associated with a location error, recording a sequence of images of the region forward of the vehicle in response to the location error exceeding a threshold error value;

identifying a landmark depicted in an image of the sequence of images with a landmark identification module of a computing system associated with the vehicle, the landmark having a landmark geographic location and a known parameter, wherein the known parameter characterizes a physical dimension of the landmark;

identifying the landmark in each of the sequence of images;

extracting a set of landmark parameters from the image with a feature extraction module of the computing system, wherein the set of landmark parameters comprises a landmark pose;

extracting a corresponding landmark pose from each of the sequence of images;

determining, at the computing system, a relative position between the vehicle and the landmark geographic location based on a comparison between the set of landmark parameters and the known parameter;

determining a temporal pattern based on the corresponding landmark pose extracted from each of the sequence of images;

determining a vehicle trajectory based on the temporal pattern, wherein the vehicle is controlled along the vehicle trajectory based on the location error, and wherein the vehicle is an autonomous vehicle;

updating, at the computing system, the global system location based on the relative position;

setting the location error to an error value below the threshold error value in response to updating the global system location; and

incrementing the location error in proportion to an elapsed time subsequent to updating the global system location.

2. The method of claim 1 , further comprising retrieving the landmark geographic location of the landmark from a remote database associated with the computing system.

3. The method of claim 2 , further comprising determining a set of unit regions based on the global system location, and retrieving the landmark geographic location from the remote database based on the set of unit regions.

4. The method of claim 2 , wherein the set of landmark parameters comprises a textual location indicator, and further comprising determining a set of unit regions based on the textual location indicator, and retrieving the landmark geographic location from the remote database based on the set of unit regions.

5. The method of claim 1 , further comprising determining a route based on an accumulation rate of the location error exceeding a threshold accumulation rate value, wherein the route is determined according to a landmark density along the route, and navigating the vehicle according to the route.

6. The method of claim 1 , further comprising determining, at the computing system, a speed value and a heading value associated with the vehicle based on the set of landmark parameters, wherein the speed value is associated with a speed error and the heading value is associated with a heading error.

7. The method of claim 6 , further comprising continuously updating the global system location based on the speed value and heading value, and continuously updating the location error based on the speed error and the heading error.

8. The method of claim 6 , further comprising measuring a wheel rotations-per-minute value, determining the speed value based on the wheel rotations-per-minute value in combination with a known wheel diameter, and determining the heading value based on the set of landmark parameters.

9. The method of claim 1 , wherein the landmark comprises a street sign having a set of corners, and wherein extracting the set of features comprises extracting a set of pixel positions of the set of corners from the image and determining the landmark pose based on the set of pixel positions.

10. The method of claim 1 , further comprising:

identifying a plurality of landmarks depicted in the image, each of the plurality of landmarks associated with a corresponding geographic location;

extracting a set of landmark parameters associated with each of the plurality of landmarks from the image;

determining a static pattern based on the set of landmark parameters associated with each of the plurality of landmarks; and

determining the global system location based on the static pattern.

Assignments (9)
RELEASE OF SECURITY INTEREST Recorded Jun 19, 2026
From: ORIX GROWTH CAPITAL, LLC
To: NAUTO, INC.
Reel/Frame 075016/0824 →
SECURITY INTEREST Recorded Aug 8, 2025
From: NAUTO, INC.
To: ORIX GROWTH CAPITAL, LLC, AS AGENT
Reel/Frame 071976/0818 →
SECURITY INTEREST Recorded Nov 10, 2022
From: NAUTO, INC.
To: SILICON VALLEY BANK
Reel/Frame 061722/0392 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE STREET ADDRESS PREVIOUSLY RECORDED AT REEL: 047977 FRAME: 0299. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jun 14, 2019
From: NAUTO GLOBAL LIMITED
To: NAUTO, INC.
Reel/Frame 049475/0814 →
CORRECTIVE ASSIGNMENT TO CORRECT THE STATE OF INCORPORATION INSIDE THE ASSIGNMENT DOCUMENT PREVIOUSLY RECORDED AT REEL: 047821 FRAME: 0958. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 21, 2018
From: NAUTO GLOBAL LIMITED
To: NAUTO, INC.
Reel/Frame 047977/0299 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 19, 2018
From: NAUTO GLOBAL INC.
To: NAUTO, INC.
Reel/Frame 047821/0958 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 30, 2017
From: NAUTO, INC.
To: NAUTO GLOBAL LIMITED
Reel/Frame 044266/0913 →
CORRECTIVE ASSIGNMENT TO CORRECT THE STATE OF INCORPORATION INSIDE THE ASSIGNMENT DOCUMENT PREVIOUSLY RECORDED AT REEL: 043535 FRAME: 0277. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT . Recorded Oct 30, 2017
From: GHADIOK, VAIBHAV; GLEESON-MAY, MICHAEL
To: NAUTO, INC.
Reel/Frame 044320/0927 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 8, 2017
From: GHADIOK, VAIBHAV; GLEESON-MAY, MICHAEL
To: NAUTO, INC.
Reel/Frame 043535/0277 →
Continuity (3)
Provisional Application 62492790 · May 1, 2017
Provisional Application 62372633 · Aug 9, 2016
Related Publication 20180045519A1 · Feb 15, 2018
Cited By (5)
US 12,353,978 US 12,481,297 US 12,577,825 US 12,670,621 US 12,684,095