IP Library Granted Patent US 11,635,758
Granted Patent B2
US 11,635,758 · App. 16/696,235 · Granted Apr 25, 2023

Risk aware executor with action set recommendations

Inventors: Kyle Hollins Wray (Fremont, CA); Stefan Witwicki (San Carlos, CA); Shlomo Zilberstein (Amherst, MA)
Assignees: Nissan North America, Inc.; The University of Massachusetts; Renault S.A.S.
G05D1/0088G01C21/3492G05D1/0212G06N7/005G05D2201/0212
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,635,758
App. No.
16/696,235
Granted
Apr 25, 2023
Kind
B2
Abstract

A method for use in traversing a vehicle transportation network by an autonomous vehicle (AV) includes traversing, by the AV, the vehicle transportation network. Traversing the vehicle transportation network includes identifying a distinct vehicle operational scenario; instantiating a first decision component instance; receiving a first set of candidate vehicle control actions from the first decision component instance; selecting an action; and controlling the AV to traverse a portion of the vehicle transportation network based on the action. The first decision component instance is an instance of a first decision component modeling the distinct vehicle operational scenario.

Claims (76)

1. A method for use in traversing a vehicle transportation network by an autonomous vehicle (AV), the method comprising:

traversing, by the AV, the vehicle transportation network, wherein traversing the vehicle transportation network comprises:

identifying a distinct vehicle operational scenario;

instantiating a first decision component instance, wherein the first decision component instance is an instance of a first decision component that models the distinct vehicle operational scenario,

wherein the first decision component instance provides a first solution to the distinct vehicle operational scenario, and

wherein the first solution is obtained offline in a first idealized situation that is decoupled from a current context of the AV;

instantiating a second decision component instance in response to receiving second sensor information corresponding to an external object

wherein the second decision component instance provides a second solution for dealing with the external object, and

wherein the second solution is obtained offline in a second idealized situation that is decoupled from the current context of the AV;

receiving a first set of candidate vehicle control actions from the first decision component instance;

receiving a second set of candidate vehicle control actions from the second decision component instance;

determining a third set of actions as an intersection between the first set of candidate vehicle control actions and the second set of candidate vehicle control actions;

selecting an action from the third set of actions;

determining whether a current value of a state factor of the current context of the AV conflicts with a value of the state factor required for the action;

response to determining that the current value of the state factor conflicts with the value required for the action, converting the action to another action; and

controlling the AV to traverse a portion of the vehicle transportation network based on the action.

2. The method of claim 1 , wherein selecting the action comprises:

determining a first priority associated with the first decision component instance and a second priority associated with the second decision component instance; and

selecting the action from the first set of candidate vehicle control actions based on a determination that the first priority is higher than the second priority.

3. The method of claim 2 , wherein a highest priority is associated with first decision components associated with higher velocity vehicles, an intermediate priority is associated with second decision components that require a rule violation, and a lowest priority is associated with other types of decision components.

4. The method of claim 1 , further comprising:

determining whether the third set of actions is empty; and

responsive to determining that the third set of actions is empty, adding a default action to the third set of actions.

5. The method of claim 1 , wherein the action is one of to stop, to advance, or to proceed.

6. The method of claim 5 , wherein controlling the AV to traverse the portion of the vehicle transportation network in accordance with the action includes:

on a condition that the action is to stop, controlling the AV to be stationary;

on a condition that the action is to advance, controlling the AV to traverse a defined cautionary distance in the vehicle transportation network at a defined cautionary rate; and

on a condition that the action is to proceed, controlling the AV to traverse the vehicle transportation network in accordance with a previously identified vehicle control action.

7. The method of claim 1 , wherein instantiating the first decision component instance includes:

on a condition that identifying the distinct vehicle operational scenario includes identifying an intersection scenario, instantiating an intersection-decision component instance, wherein the intersection-decision component instance is an instance of an intersection-decision component modeling the intersection scenario;

on a condition that identifying the distinct vehicle operational scenario includes identifying a pedestrian scenario, instantiating a pedestrian-decision component instance, wherein the pedestrian-decision component instance is an instance of a pedestrian-decision component modeling the pedestrian scenario; and

on a condition that identifying the distinct vehicle operational scenario includes identifying a lane-change scenario, instantiating a lane-change-decision component instance, wherein the lane-change-decision component instance is an instance of a lane-change-decision component modeling the lane-change scenario.

8. The method of claim 1 , wherein the state states factor relates to at least one of a first state of a sensor of the AV, a second state of an actuator of the AV, a third state of availability of a tele-operator, or a fourth state relating to an autonomy competence level of the AV.

9. An autonomous vehicle comprising:

a processor configured to execute instructions stored on a non-transitory computer readable medium to:

instantiating respective decision component instances by instructions to:

responsive to identifying a distinct vehicle operational of an intersection scenario, instantiate an intersection-decision component instance,

wherein the intersection-decision component instance is an instance of an intersection- decision component modeling the intersection scenario, and

wherein the intersection-decision component instance provides a first solution that is obtained offline in a first idealized situation that is decoupled from a current context of the autonomous vehicle;

responsive to identifying a distinct vehicle operational of a pedestrian scenario, instantiate a pedestrian-decision component instance,

wherein the pedestrian-decision component instance is an instance of a pedestrian-decision component modeling the pedestrian scenario, and

wherein the instantiate a pedestrian-decision component instance provides a second solution that is obtained offline in a second idealized situation that is decoupled from the current context of the autonomous vehicle; and

responsive to identifying a distinct vehicle operational of a lane-change scenario, instantiating a lane-change-decision component instance,

wherein the lane-change-decision component instance is an instance of a lane-change-decision component modeling the lane-change scenario, and

wherein the lane-change-decision component instance provides a third solution that is obtained offline in a third idealized situation that is decoupled from the current context of the autonomous vehicle;

receive respective sets of candidate actions from the respective decision component instances, wherein each decision component instance is an instance of a decision component modeling a distinct vehicle operational scenario;

identify an intersecting set of actions as an intersection of the respective sets of candidate actions;

select an action from the intersecting set of actions based on respective priorities of the respective decision components;

determine whether a current value of a state factor of the current context of the AV conflicts with a value of the state factor required for the action;

response to determining that the current value of the state factor conflicts with the value required for the action, convert the action to another action; and

control the autonomous vehicle to traverse a portion of a vehicle transportation network based on the action.

10. The autonomous vehicle of claim 9 , wherein a highest priority is associated with first decision components associated with higher velocity external objects, an intermediate priority is associated with second decision components that require a rule violation, and a lowest priority is associated with other types of decision components.

11. The autonomous vehicle of claim 9 , wherein the processor is further configured to execute instructions stored in the non-transitory computer readable medium to:

responsive to determining that the intersecting set of actions is empty, add a default action to the intersecting set of actions.

12. The autonomous vehicle of claim 11 , wherein the action is selected according to a Partially Observable Markov Decision Process model.

13. A method for use in traversing a vehicle transportation network by an autonomous vehicle, the method comprising:

receiving a first set of candidate actions from a first decision component instance and a second set of candidate actions from a second decision component instance,

wherein the first decision component instance is a first instance of a first decision component modeling a first distinct vehicle operational scenario,

wherein the first decision component instance provides a first solution to the first distinct vehicle operational scenario,

wherein the first solution is obtained offline in a first idealized situation that is decoupled from a current context of an autonomous vehicle (AV),

wherein the second decision component instance is a second instance of a second decision component modeling a second distinct vehicle operational scenario,

wherein the second decision component instance provides a second solution to the second distinct vehicle operational scenario, and

wherein the second solution is obtained offline in a second idealized situation that is decoupled from the current context of the AV;

identifying an intersecting set of actions as an intersection of the first set of candidate actions and the second set of candidate actions;

determining whether the intersecting set of actions is empty;

responsive to determining that the intersecting set of actions is empty, adding a default action to the intersecting set of actions;

selecting the action is conditioned on a context of state vectors associated with the autonomous vehicle, wherein selecting the action comprises:

determining whether a current value of a state factor of the current context of the AV conflicts with a value of the state factor required for the action; and

response to determining that the current value of the state factor conflicts with the value required for the action, converting the action to another action; and

controlling the autonomous vehicle to traverse a portion of the vehicle transportation network based on the action.

14. The method of claim 13 , wherein selecting the action is conditioned on a first priority of the first decision component and a second priority of the second decision component.

15. The method of claim 13 , wherein the action is at least one of to stop, to advance, to proceed, to pass-on-left, or to pass-on-right.

16. The method of claim 4 , wherein the default action is a stop action.

17. The method of claim 13 , further comprising:

determining whether the intersecting set of actions is empty; and

responsive to determining that the intersecting set of actions is empty, adding a default action to the intersecting set of actions.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 8, 2023
From: NISSAN NORTH AMERICA, INC.
To: NISSAN MOTOR CO., LTD.
Reel/Frame 064525/0488 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 18, 2020
From: NISSAN NORTH AMERICA, INC.
To: NISSAN NORTH AMERICA, INC.; RENAULT S.A.S.
Reel/Frame 051844/0750 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 4, 2020
From: ZILBERSTEIN, SHLOMO
To: THE UNIVERSITY OF MASSACHUSETTS
Reel/Frame 051711/0265 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 4, 2020
From: WITWICKI, STEFAN
To: NISSAN NORTH AMERICA, INC.
Reel/Frame 051711/0370 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 4, 2020
From: WRAY, KYLE HOLLINS
To: NISSAN NORTH AMERICA, INC.; THE UNIVERSITY OF MASSACHUSETTS
Reel/Frame 051711/0476 →
Continuity (1)
Related Publication 20210157315A1 · May 27, 2021
Cited By (4)
US 12,198,080 US 12,391,274 US 12,643,571 US 12,691,905