IP Library Granted Patent US 11,300,957
Granted Patent B2
US 11,300,957 · App. 16/727,038 · Granted Apr 12, 2022

Multiple objective explanation and control interface design

Inventors: Kyle Hollins Wray (Fremont, CA); Stefan Witwicki (San Carlos, CA); Shlomo Zilberstein (Amherst, MA)
Assignees: Nissan North America, Inc.; The University of Massachusetts; Renault S.A.S.
G05D1/0088B60W30/0953G08G1/166G05D1/0217G08G1/167
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,300,957
App. No.
16/727,038
Granted
Apr 12, 2022
Kind
B2
Abstract

A vehicle traversing a vehicle transportation network may use a scenario-specific operational control evaluation model instance. A multi-objective policy for the model is received, wherein the policy includes at least a first objective, a second objective, and a priority of the first objective relative to the second objective. A representation of the policy (e.g., the first objective, the second objective, and the priority) is generated using a user interface. Based on feedback to the user interface, a change to the multi-objective policy for the scenario-specific operational control evaluation model is received. The change is to the first objective, the second objective, the priority, of some combination thereof. Then, for determining a vehicle control action for traversing the vehicle transportation network, an updated multi-objective policy for the scenario-specific operational control evaluation model is generated to include the change to the policy.

Claims (89)

1. A method for use with a vehicle in traversing a vehicle transportation network, the method comprising:

receiving, from an instance of a scenario-specific operational control evaluation model of a distinct vehicle operational scenario, a multi-objective policy for the scenario-specific operational control evaluation model, wherein the multi-objective policy includes at least a first objective, a second objective, and a priority of the first objective relative to the second objective;

generating, using a user interface, a representation of the first objective, the second objective, and the priority of the first objective relative to the second objective;

receiving, based on feedback to the user interface, a change to the multi-objective policy for the scenario-specific operational control evaluation model, the change including at least one of a change of the first objective, a change of the second objective, or a change in the priority of the first objective relative to the second objective; and

generating, for determining a vehicle control action for traversing the vehicle transportation network, an updated multi-objective policy for the scenario-specific operational control evaluation model to include the change to the multi-objective policy.

2. The method of claim 1 , wherein the priority of the first objective relative to the second objective comprises a preference for the first objective over the second objective and a buffer value indicating a constraint applied to the preference within the scenario-specific operational control evaluation model.

3. The method of claim 2 , wherein:

the buffer value indicates how many units, of a defined measurement unit, that the scenario-specific operational control evaluation model will allow deviation from the second objective in favor of the first objective, and

a change in the priority of the first objective relative to the second objective comprises a change to the buffer value.

4. The method of claim 3 , wherein the defined measurement unit comprises seconds.

5. The method of claim 1 , wherein:

the user interface comprises a graphical user interface,

the representation comprises a first object representing the first objective and a second object representing the second objective, and

an arrangement of the first object and the second object within the representation on the graphical user interface indicates the priority of the first objective relative to the second objective.

6. The method of claim 5 , wherein:

the multi-objective policy includes a third objective, and a priority of the third objective relative to at least one of the first objective or the second objective,

the representation comprises a third object representing the third objective, and

the arrangement within the representation on the graphical user interface further indicates the priority of the third objective relative to at least one of the first objective or the second objective.

7. The method of claim 6 , wherein:

the priority of the first objective relative to the second objective includes a first buffer value that indicates how many units, of a defined measurement unit, that the scenario-specific operational control evaluation model will allow deviation from the second objective in favor of the first objective,

the priority of the third objective relative to at least one of the first objective or the second objective includes a second buffer value that indicates one of:

how many units, of the defined measurement unit, that the scenario-specific operational control evaluation model will allow deviation from the third objective in favor of the first objective, or

how many units, of the defined measurement unit, that the scenario-specific operational control evaluation model will allow deviation from the third objective in favor of the second objective.

8. The method of claim 7 , wherein:

the second buffer value indicates how many units, of the defined measurement unit, that the scenario-specific operational control evaluation model will allow deviation from the third objective in favor of the second objective, and

the arrangement comprises:

the first object, the second object, and the third object ordered within a list such that the first object is located at a bottom of the list, the second object is located above the first object within the list, and the third object is located above the second object within the list.

9. The method of claim 8 , wherein:

the first object comprises a descriptor of the first objective,

the second object comprises a descriptor of the second objective, identification of the first buffer value, and a first embedded object that, responsive to contact, changes the first buffer value, and

the third object comprises a descriptor of the third objective, identification of the second buffer value, and a second embedded object that, responsive to contact, changes the second buffer value.

10. The method of claim 7 , wherein:

the second buffer value indicates how many units, of the defined measurement unit, that the scenario-specific operational control evaluation model will allow deviation from the third objective in favor of the first objective, and

the arrangement comprises:

the second object and the third object adjacent to each other in a row in parallel with a top of the graphical user interface, and the first object located below the second object and the third object.

11. The method of claim 10 , wherein:

the first object comprises a descriptor of the first objective,

the second object comprises a descriptor of the second objective, identification of the first buffer value, and a first embedded object that, responsive to contact, changes the first buffer value, and

the third object comprises a descriptor of the third objective, identification of the second buffer value, and a second embedded object that, responsive to contact, changes the second buffer value.

12. The method of claim 10 , wherein:

the first object comprises a descriptor of the first objective,

the second object comprises a descriptor of the second objective,

the third object comprises a descriptor of the third objective, and

the representation includes:

an arrow directed from the second object to the first object,

a fourth object that identifies the first buffer value and that, responsive to contact, changes the first buffer value,

an arrow directed from the third object to the first object, and

a fifth object that identifies the second buffer value and that, responsive to contact, changes the second buffer value.

13. An apparatus for use with a vehicle traversing a vehicle transportation network, the apparatus comprising:

a user interface; and

a processor configured to:

receive, from an instance of a scenario-specific operational control evaluation model of a distinct vehicle operational scenario, a multi-objective policy for the scenario-specific operational control evaluation model, wherein the multi-objective policy includes at least a first objective, a second objective, and a priority of the first objective relative to the second objective;

generate, using the user interface, a representation of the first objective, the second objective, and the priority of the first objective relative to the second objective;

receive, based on feedback to the user interface, a change to the multi-objective policy for the scenario-specific operational control evaluation model, the change including at least one of a change of the first objective, a change of the second objective, or a change in the priority of the first objective relative to the second objective; and

generate, for determining a vehicle control action for traversing the vehicle transportation network, an updated multi-objective policy for the scenario-specific operational control evaluation model to include the change to the multi-objective policy.

14. The apparatus of claim 13 , further comprising:

a wireless communications device; and

a memory storing the multi-objective policy, wherein the processor is configured to:

receive the multi-objective policy from the memory responsive to a request, through the wireless communications device, for assistance from the vehicle,

present the vehicle control action that is determined using the updated multi-objective policy using the user interface, and

transmit an instruction to the vehicle to execute the vehicle control action.

15. The apparatus of claim 14 , wherein the processor is configured to:

store the updated multi-objective policy as the multi-objective policy for a subsequent instance of the scenario-specific operational control evaluation model of the distinct vehicle operational scenario.

16. The apparatus of claim 13 , wherein the user interface comprises a speaker, and the representation is a sentence that identifies the first objective, the second objective, and the priority of the first objective relative to the second objective.

17. The apparatus of claim 16 , wherein the user interface comprises a microphone, and the processor is configured to receive the change from a verbal command to the microphone.

18. The apparatus of claim 13 , wherein:

the user interface comprises a graphical user interface,

the representation comprises a first object displayed on the graphical user interface representing the first objective and a second object displayed on the graphical user interface representing the second objective,

the first object is a drag-and-drop object and includes a descriptor of the first objective,

the second object is a drag-and-drop object and includes a descriptor of the second objective,

an arrangement of the first object and the second object within the representation on the graphical user interface indicates the priority of the first objective relative to the second objective,

the priority of the first objective relative to the second objective includes a first buffer value that indicates how many units, of a defined measurement unit, that the scenario-specific operational control evaluation model will allow deviation from the second objective in favor of the first objective, and

the representation comprises a third object associated with the first buffer value, wherein the third object is one of a pull-down menu or a slider that changes the first buffer value responsive to contact with the graphical user interface.

19. The apparatus of claim 13 , wherein:

the first objective is a first candidate objective of a group of candidate objectives,

the second objective is a second candidate objective of the group of candidate objectives, and

the change to the multi-objective policy for the scenario-specific operational control evaluation model comprises at least one of changing the first objective by changing the first candidate objective to another candidate objective of the group of candidate objectives, changing the second objective to another candidate objective of the group of candidate objectives, adding a third objective from the group of candidate objectives with a priority of the third objective relative to at least one of the first objective or the second objective, or the change in the priority of the first objective relative to the second objective.

20. An autonomous vehicle, comprising:

a user interface comprising at least one of:

a graphical user interface; or

a microphone and a speaker; and

a processor configured to:

receive, from an instance of a scenario-specific operational control evaluation model of a distinct vehicle operational scenario, a multi-objective policy for the scenario-specific operational control evaluation model, wherein:

the multi-objective policy includes at least a first objective, a second objective, and a priority of the first objective relative to the second objective,

the priority of the first objective relative to the second objective comprises a preference for the first objective over the second objective and a buffer value indicating a constraint applied to the preference within the scenario-specific operational control evaluation model, and

the buffer value indicates how many units, of a defined measurement unit, that the scenario-specific operational control evaluation model will allow deviation from the second objective in favor of the first objective;

generate, using the user interface, a representation of the first objective, the second objective, and the priority of the first objective relative to the second objective;

receive, based on feedback to the user interface, a change to the multi-objective policy for the scenario-specific operational control evaluation model, the change including at least one of a change of the first objective, a change of the second objective, or a change in the priority of the first objective relative to the second objective; and

generate, for determining a vehicle control action for traversing a vehicle transportation network, an updated multi-objective policy for the scenario-specific operational control evaluation model to include the change to the multi-objective policy.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 28, 2022
From: NISSAN NORTH AMERICA, INC.
To: NISSAN MOTOR CO., LTD.
Reel/Frame 060656/0554 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 18, 2020
From: NISSAN NORTH AMERICA, INC.
To: NISSAN NORTH AMERICA, INC.; RENAULT S.A.S.
Reel/Frame 051844/0750 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 4, 2020
From: ZILBERSTEIN, SHLOMO
To: THE UNIVERSITY OF MASSACHUSETTS
Reel/Frame 051712/0577 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 4, 2020
From: WITWICKI, STEFAN
To: NISSAN NORTH AMERICA, INC.
Reel/Frame 051712/0654 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 4, 2020
From: WRAY, KYLE HOLLINS
To: NISSAN NORTH AMERICA, INC.; THE UNIVERSITY OF MASSACHUSETTS
Reel/Frame 051712/0807 →
Continuity (1)
Related Publication 20210200208A1 · Jul 1, 2021
Cited By (10)
US 12,240,494 US 12,296,849 US 12,319,313 US 12,371,067 US 12,394,311 US 12,441,364 US 12,488,685 US 12,559,132 US 12,559,143 US 12,594,961