IP Library Granted Patent US 11,981,349
Granted Patent B2
US 11,981,349 · App. 17/178,464 · Granted May 14, 2024

Behavior planning for autonomous vehicles

Inventors: David Nister (Bellevue, WA); Yizhou Wang (San Jose, CA); Julia Ng (San Jose, CA); Rotem Aviv (San Diego, CA); Seungho Lee (San Jose, CA); Joshua John Bialkowski (San Mateo, CA); Hon Leung Lee (Bellevue, WA); Hermes Lanker (Zürich, CH); Raul Correal Tezanos (Santa Clara, CA); Zhenyi Zhang (San Jose, CA); Nikolai Smolyanskiy (Seattle, WA); Alexey Kamenev (Carlsbad, CA); Ollin Boer Bohan (Redmond, WA); Anton Vorontsov (San Jose, CA); Miguel Sainz Serra (Palo Alto, CA); Birgit Henke (Seattle, WA)
Assignee: NVIDIA Corporation
B60W60/0011B60W50/0097G05D1/0212G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,981,349
App. No.
17/178,464
Granted
May 14, 2024
Kind
B2
Abstract

Embodiments of the present disclosure relate to behavior planning for autonomous vehicles. The technology described herein selects a preferred trajectory for an autonomous vehicle based on an evaluation of multiple hypothetical trajectories by different components within a planning system. The various components provide an optimization score for each trajectory according to the priorities of the component and scores from multiple components may form a final optimization score. This scoring system allows the competing priorities (e.g., comfort, minimal travel time, fuel economy) of different components to be considered together. In examples, the trajectory with the best combined score may be selected for implementation. As such, an iterative approach that evaluates various factors may be used to identify an optimal or preferred trajectory for an autonomous vehicle when navigating an environment.

Claims (41)

1. A method comprising:

computing time scores assigned to corresponding potential future locations of a machine based at least on analyzing geographic information of a geographic area associated with the machine;

based at least on the computing of the time scores, selecting a nominal rail based at least on rating the nominal rail using a first series of the time scores that corresponds to the nominal rail;

based at least on the selecting, generating, using the nominal rail, a plurality of possible trajectories associated with the machine completing an autonomous machine action associated with the nominal rail;

based at least on the generating, rating at least one trajectory of the plurality of possible trajectories based at least on adjusting a second series of the time scores that corresponds to the at least one trajectory to customize the second series of the time scores to the at least one trajectory;

selecting a trajectory for implementation based at least on the rating of the at least one trajectory; and

implementing the trajectory using the machine.

2. The method of claim 1 , wherein the selecting of the nominal rail is from a plurality of nominal rails based at least on the rating of the nominal rail relative to a second nominal rail of the plurality of nominal rails.

3. The method of claim 1 , wherein the method further includes:

annotating nodes of a lane graph, that represent the corresponding potential future locations, with the time scores;

determining, using the annotated nodes of the lane graph, the first series of the time scores that correspond to the nominal rail; and

determining, using the annotated nodes of the lane graph, a second series of the time scores that correspond to a second nominal rail, wherein the selecting of the nominal rail is based at least on the first series of the time scores and the second series of the time scores.

4. The method of claim 1 , wherein the first series of the time scores are along the nominal rail over a first planning horizon, and the rating of the at least one trajectory using the second series of the time scores is over a second planning horizon that is smaller than the first planning horizon.

5. The method of claim 1 , wherein the time scores are determined based at least on analyzing a first geographic map, and the adjusting is based at least on analyzing a second geographic map that has a higher level of detail than the first geographic map and covers a smaller geographic area than the first geographic map.

6. The method of claim 1 , wherein the at least one trajectory includes the trajectory and the selecting of the trajectory is from the plurality of possible trajectories using, for each given trajectory of the plurality of possible trajectories, a respective series of the time scores adjusted to correspond to the given trajectory.

7. The method of claim 1 , wherein the rating includes computing an optimization score for the at least one trajectory, and the adjusting adds time to a time of travel represented by the second series of time scores as a penalty to produce the optimization score.

8. The method of claim 1 , wherein the time scores are associated with corresponding graph nodes of a lane graph, and the nominal rail comprises a directed path traversing a subset of the graph nodes.

9. A method comprising:

computing time scores assigned to corresponding potential future locations of a machine;

based at least on the computing of the time scores, selecting a nominal longitudinal rail corresponding to an autonomous machine action for the machine based at least on rating the nominal longitudinal rail using a first series of the time scores that corresponds to the nominal longitudinal rail;

based at least on the selecting, generating a plurality of possible trajectories for the autonomous machine action based at least on laterally varying the nominal longitudinal rail;

based at least on the generating, computing an optimization score for at least one trajectory of the plurality of possible trajectories based at least on adjusting a second series of the time scores that corresponds to the at least one trajectory to customize the optimization score to the at least one trajectory;

determining an individual trajectory using the optimization score; and

implementing the individual trajectory for the machine.

10. The method of claim 9 , further comprising generating longitudinal limits for the autonomous machine action using a scene prediction that includes a predicted future object location for an object detected by one or more sensors corresponding to the machine.

11. The method of claim 9 , wherein the adjusting includes computing a magnitude of an adjustment to the second series of the time scores based at least on a time to a destination for the machine.

12. The method of claim 9 , the selecting includes selecting the autonomous machine action for the machine using a lane graph annotated with the time scores as selection input data.

13. The method of claim 9 , further comprising selecting a route between a starting point and a route destination using a geographic data set for a geographic area as input.

14. The method of claim 9 , wherein the laterally varying the nominal longitudinal rail includes maintaining curvature of the nominal longitudinal rail in the plurality of possible trajectories.

15. The method of claim 9 , wherein each time score of the time scores quantifies an overall value of the machine reaching a corresponding location of the potential future locations.

16. The method of claim 9 , wherein the laterally varying the nominal longitudinal rail includes introducing longitudinal variations of the nominal longitudinal rail to the plurality of possible trajectories within longitudinal limits corresponding to the nominal longitudinal rail.

17. A processor comprising:

one or more processing units to determine a trajectory for a machine using time scores assigned to corresponding potential future locations of the machine based at least on selecting a nominal rail, the nominal rail being selected based at least on:

rating the nominal rail using a first series of the time scores that corresponds to the nominal rail,

based at least the selecting, generating at least one trajectory, and

based at least on the generating, rating the at least one trajectory based at least on adjusting a second series of the time scores that corresponds to the at least one trajectory to customize the second series of the time scores to the at least one trajectory.

18. The processor of claim 17 , wherein the one or more processing units are further to generate an annotated lane graph comprising the time scores on graph nodes representing positions in one or more lanes, wherein each of the time scores corresponds to a travel time to a route destination.

19. The processor of claim 17 , wherein the one or more processing units are further to:

select, using the rating, the trajectory for a starting parameter set used to generate additional trajectories for evaluation; and

evaluate the additional trajectories using the time scores to determine the trajectory for the machine.

20. The processor of claim 17 , wherein the adjusting includes adding one or more time penalties to the second series of time scores.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 1, 2021
From: NISTER, DAVID; WANG, YIZHOU; NG, JULIA; AVIV, ROTEM; BIALKOWSKI, JOSHUA JOHN; LEE, HON LEUNG; LEE, SEUNGHO; LANKER, HERMES; TEZANOS, RAUL CORREAL; ZHANG, ZHENYI; SMOLYANSKIY, NIKOLAI; KAMENER, ALEXEY; BOHAN, OLLIN BOER; VORONTSOV, ANTON; SERRA, MIGUEL SAINZ; HENKE, BIRGIT
To: NVIDIA CORPORATION
Reel/Frame 055443/0723 →
Continuity (2)
Provisional Application 62978309 · Feb 19, 2020
Related Publication 20210253128A1 · Aug 19, 2021
Cited By (3)
US 12,304,509 US 12,509,072 US 12,637,106