IP Library Granted Patent US 11,620,831
Granted Patent B2
US 11,620,831 · App. 16/862,546 · Granted Apr 4, 2023

Register sets of low-level features without data association

Inventors: Jeffrey Michael Walls (Ann Arbor, MI); Ryan W. Wolcott (Ann Arbor, MI)
Assignee: TOYOTA RESEARCH INSTITUTE, INC.
G06V20/56G01C21/3453G01C21/3635G06K9/6232G06K9/6272
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,620,831
App. No.
16/862,546
Granted
Apr 4, 2023
Kind
B2
Abstract

Systems and methods for localization of a subject vehicle may include: a plurality of sensors to gather data regarding features in an environment of the subject vehicle; a feature map module to generate a feature map using data gathered from one or more of the plurality of sensors; a feature raster module to generate a feature raster image from the feature map; a score computation module to register a plurality of transformations of the feature raster image with a map feature raster image, to generate a cost function for the registrations of the plurality of transformations, and to determine the transformation with the lowest cost function; and the score computation module using the transformation with the lowest cost function to determine localization of the subject vehicle.

Claims (51)

1. A system for localization of a subject vehicle, comprising:

a plurality of sensors to gather data regarding features in an environment of the subject vehicle;

a memory; and

one or more processors that are configured to execute machine readable instructions stored in the memory for performing a method comprising:

receiving data from the plurality of sensors;

generating a three-dimensional feature map using the data, wherein the three-dimensional feature map comprises a plurality of features, wherein the plurality of features comprises at least one of landmarks, telephone posts, and light posts;

extracting a feature raster image from the feature map, wherein the feature raster image comprises a binary raster map;

registering a plurality of transformations of the feature raster image;

generating a plurality of cost functions, wherein one cost function is associated with each transformation;

generating a score associated with each cost function, wherein a higher score demonstrates a lower cost;

determining the transformation with the lowest cost based on the scores and selecting the transformation with the lowest cost for localization of the subject vehicle; and

determining localization of the subject vehicle based on the transformation with the lowest cost.

2. The system of claim 1 , wherein registering a plurality of transformations comprises registering low level features without data association.

3. The system of claim 2 , wherein the low level features relate to non-zero elements for at least one of 3D points, lines and planes.

4. The system of claim 1 , wherein registration is performed using only nonzero elements of the feature raster.

5. A system for vehicle localization using map data from multiple sources, the system comprising:

a non-transitory memory configured to store instructions;

at least one processor configured to execute the instructions to:

generate a set of vehicle map data using vehicle sensors;

project features of the set of vehicle map data into discretized feature representation;

transform the discretized feature representation from a vehicle frame of reference to an external map frame of reference to generate a transformed discretized three-dimensional feature representation, wherein the transformed discretized three-dimensional feature representation comprises a plurality of features, wherein the plurality of features comprises at least one of landmarks, telephone posts, and light posts

register the transformed discretized feature representation with a map feature representation of a set of external map data and determine a cost function associated with the registration, wherein the map feature representation comprises a binary raster map;

repeat the registration for a plurality of transformations of the discretized feature representation and determine a plurality of cost functions, wherein one cost function is associated with each of the plurality of transformations;

generate a score associated with each cost function, wherein a higher score demonstrates a lower cost; and

select the transformation with the lowest cost for localization of the subject vehicle.

6. The system of claim 5 , wherein projecting features of the map data into discretized feature representation comprises creating a feature raster comprising a grid of pixels identifying features in the map data and transforming the discretized feature representation comprises transforming the feature raster.

7. The system of claim 6 , wherein the processor is configured to further execute an instruction to determine a discrete map feature representation of a set of external map data received from a source external to the vehicle.

8. The system of claim 7 , wherein registering the transformed discretized feature representation with a map feature representation of a set of external map data comprises registering the transformed feature raster with a feature raster of the external map data.

9. The system of claim 5 , wherein registering the transformed discretized feature representation with a map feature representation of a set of external map data comprises performing template matching.

10. A method for vehicle localization, comprising:

generating a set of vehicle map data using vehicle sensors;

projecting features of the set of vehicle map data into discretized feature representation;

transforming the discretized feature representation from a vehicle frame of reference to an external map frame of reference to generate a transformed discretized three-dimensional feature representation, wherein the transformed discretized three-dimensional feature representation comprises a plurality of features, wherein the plurality of features comprises at least one of landmarks, telephone posts, and light posts;

registering the transformed discretized feature representation with a map feature representation of a set of external map data and determine a cost function associated with the registration, wherein the map feature representation comprises a binary raster map;

repeating the registration for a plurality of transformations of the discretized feature representation and determine a plurality of cost functions, wherein one cost function is associated with each of the plurality of transformations;

generating a score associated with each cost function, wherein a higher score demonstrates a lower cost; and

selecting the transformation with the lowest cost for localization of the subject vehicle.

11. The method of claim 10 , wherein projecting features of the map data into discretized feature representation comprises creating a feature raster comprising a grid of pixels identifying features in the map data and transforming the discretized feature representation comprises transforming the feature raster.

12. The method of claim 11 , further comprising determining a discrete map feature representation of a set of external map data received from a source external to the vehicle.

13. The method of claim 12 , wherein registering the transformed discretized feature representation with a map feature representation of a set of external map data comprises registering the transformed feature raster with a feature raster of the external map data.

14. The method of claim 10 , wherein registering the transformed discretized feature representation with a map feature representation of a set of external map data comprises performing template matching.

15. A method of localization of a subject vehicle, comprising:

receiving data regarding features in an environment of the subject vehicle;

generating a three-dimensional feature map using data gathered from one or more of the plurality of sensors, wherein the three-dimensional feature map comprises a plurality of features, wherein the plurality of features comprises at least one of landmarks, telephone posts, and light posts;

extracting a feature raster image from the feature map wherein the feature raster image comprises a binary raster map;

registering a plurality of transformations of the feature raster image, to generate a plurality of cost functions, wherein one cost function is associated with each of the plurality of transformations, to generate a score associated with each cost function, wherein a higher score demonstrates a lower cost, and to determine the transformation with the lowest cost;

selecting the transformation with the lowest cost for localization of the subject vehicle; and determining localization of the subject vehicle, based on the transformation with the lowest cost.

16. The method of claim 15 , wherein registering a plurality of transformations comprises registering low level features without data association.

17. The system of claim 16 , wherein the low level features relate to non-zero elements for at least one of 3D points, lines and planes.

18. The system of claim 15 wherein registration is performed using only nonzero elements of the feature raster.

19. The system of claim 1 , wherein the plurality of features comprises at least one of traffic elements, lane markings, road boundaries.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 18, 2023
From: TOYOTA RESEARCH INSTITUTE, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 063681/0081 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2020
From: WALLS, JEFFREY MICHAEL; WOLCOTT, RYAN W.
To: TOYOTA RESEARCH INSTITUTE, INC.
Reel/Frame 052531/0420 →
Continuity (1)
Related Publication 20210342599A1 · Nov 4, 2021
Cited By (1)
US 12,327,398