IP Library › Granted Patent US 11,181,383
Granted Patent B2
US 11,181,383 · App. 16/406,809 · Granted Nov 23, 2021

Systems and methods for vehicular navigation and localization

Inventors: Alexander Amini (Cambridge, MA); Guy Rosman (Newton, MA); Sertac Karaman (Cambridge, MA); Daniela Rus (Weston, MA)
Assignees: Toyota Research Institute, Inc.; Massachusetts Institute of Technology
G01C21/28B60W10/04B60W10/18B60W10/20B60W30/18B60W40/10B60W2400/00B60W2420/42B60W2420/52B60W2420/54B60W2710/18B60W2710/20B60W2720/106
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,181,383
App. No.
16/406,809
Granted
Nov 23, 2021
Kind
B2
Abstract

Systems and methods described herein relate to vehicular navigation and localization. One embodiment extracts perceptual features from sensor data; extracts unrouted-map features from unrouted map data; combines the perceptual features and the unrouted-map features to produce first combined features data; outputs, based at least in part on the first combined features data, parameters of a probability distribution for one or more steering trajectories that are available to a vehicle; and performs a localization of the vehicle based, at least in part, on the parameters of the probability distribution.

Claims (56)

1. A system for vehicular navigation and localization, the system comprising:

one or more sensors to output sensor data;

one or more processors; and

a memory communicably coupled to the one or more processors and storing:

a computation module including instructions that when executed by the one or more processors cause the one or more processors to:

extract perceptual features from the sensor data;

extract unrouted-map features from unrouted map data;

combine the perceptual features and the unrouted-map features to produce first combined features data;

output, based at least in part on the first combined features data, parameters of a probability distribution for one or more steering trajectories that are available to a vehicle;

extract routed-map features from routed map data;

combine the routed-map features with the first combined features data to produce second combined features data; and

output, based at least in part on the second combined features data, a deterministic steering control signal corresponding to a particular one of the one or more steering trajectories;

a control module including instructions that when executed by the one or more processors cause the one or more processors to control steering of the vehicle based, at least in part, on the deterministic steering control signal; and

a localization module including instructions that when executed by the one or more processors cause the one or more processors to perform a localization of the vehicle based, at least in part, on the parameters of the probability distribution and an updated posterior estimate of a pose of the vehicle, wherein computing the updated posterior estimate of the pose of the vehicle includes performing double marginalization over the deterministic steering control signal and the pose of the vehicle.

2. The system of claim 1 , wherein the vehicle is an ego vehicle and the computation module includes further instructions that when executed by the one or more processors cause the one or more processors to:

predict that the ego vehicle will travel along a particular one of the one or more steering trajectories based, at least in part, on the parameters of the probability distribution; and

control one or more vehicle systems of the ego vehicle in response to the particular one of the one or more steering trajectories.

3. The system of claim 1 , wherein the vehicle is a road agent external to an ego vehicle and the computation module includes further instructions that when executed by the one or more processors cause the one or more processors to:

predict that the road agent will travel along a particular one of the one or more steering trajectories based, at least in part, on the parameters of the probability distribution; and

control one or more vehicle systems of the ego vehicle in response to the particular one of the one or more steering trajectories.

4. The system of claim 1 , wherein the localization module includes instructions to perform the localization of the vehicle by distinguishing a particular location in a set of locations from the other locations in the set of locations based, at least in part, on the parameters of the probability distribution.

5. The system of claim 1 , wherein the probability distribution is for a steering control command over inverse curvature conditioned on the perceptual features and the unrouted-map features.

6. The system of claim 5 , wherein the probability distribution is a Gaussian Mixture Model.

7. The system of claim 1 , wherein the localization module includes further instructions to:

detect that at least one of the localization of the vehicle and the unrouted map data is inconsistent with the sensor data; and

correct the localization of the vehicle based, at least in part, on the parameters of the probability distribution.

8. A non-transitory computer-readable medium for vehicular navigation and localization and storing instructions that when executed by one or more processors cause the one or more processors to:

extract perceptual features from sensor data;

extract unrouted-map features from unrouted map data;

combine the perceptual features and the unrouted-map features to produce first combined features data;

output, based at least in part on the first combined features data, parameters of a probability distribution for one or more steering trajectories that are available to a vehicle;

extract routed-map features from routed map data;

combine the routed-map features with the first combined features data to produce second combined features data;

output, based at least in part on the second combined features data, a deterministic steering control signal corresponding to a particular one of the one or more steering trajectories;

control steering of the vehicle based, at least in part, on the deterministic steering control signal; and

perform a localization of the vehicle based, at least in part, on the parameters of the probability distribution and an updated posterior estimate of a pose of the vehicle, wherein computing the updated posterior estimate of the pose of the vehicle includes performing double marginalization over the deterministic steering control signal and the pose of the vehicle.

9. A method of vehicular navigation and localization, the method comprising:

extracting perceptual features from sensor data;

extracting unrouted-map features from unrouted map data;

combining the perceptual features and the unrouted-map features to produce first combined features data;

outputting, based at least in part on the first combined features data, parameters of a probability distribution for one or more steering trajectories that are available to a vehicle;

extracting routed-map features from routed map data;

combining the routed-map features with the first combined features data to produce second combined features data;

outputting, based at least in part on the second combined features data, a deterministic steering control signal corresponding to a particular one of the one or more steering trajectories;

controlling steering of the vehicle based, at least in part, on the deterministic steering control signal; and

performing a localization of the vehicle based, at least in part, on the parameters of the probability distribution and an updated posterior estimate of a pose of the vehicle, wherein computing the updated posterior estimate of the pose of the vehicle includes performing double marginalization over the deterministic steering control signal and the pose of the vehicle.

10. The method of claim 9 , wherein the vehicle is an ego vehicle and the method further comprises:

predicting that the ego vehicle will travel along a particular one of the one or more steering trajectories based, at least in part, on the parameters of the probability distribution; and

controlling one or more vehicle systems of the ego vehicle in response to the particular one of the one or more steering trajectories.

11. The method of claim 9 , wherein the vehicle is a road agent external to an ego vehicle and the method further comprises:

predicting that the road agent will travel along a particular one of the one or more steering trajectories based, at least in part, on the parameters of the probability distribution; and

controlling one or more vehicle systems of the ego vehicle in response to the particular one of the one or more steering trajectories.

12. The method of claim 9 , wherein performing the localization of the vehicle includes distinguishing a particular location in a set of locations from the other locations in the set of locations based, at least in part, on the parameters of the probability distribution.

13. The method of claim 9 , further comprising:

detecting that at least one of the localization of the vehicle and the unrouted map data is inconsistent with the sensor data; and

correcting the localization of the vehicle based, at least in part, on the parameters of the probability distribution.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2022
From: TOYOTA RESEARCH INSTITUTE, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 058557/0697 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2019
From: AMINI, ALEXANDER; KARAMAN, SERTAC; RUS, DANIELA
To: MASSACHUSETTS INSTITUTE OF TECHNOLOGY
Reel/Frame 050756/0312 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2019
From: ROSMAN, GUY
To: TOYOTA RESEARCH INSTITUTE, INC.
Reel/Frame 050764/0050 →
Continuity (2)
Provisional Application 62731896 · Sep 15, 2018
Related Publication 20200088525A1 · Mar 19, 2020
Cited By (2)
US 12,202,515 US 12,377,839