IP Library Granted Patent US 10,737,702
Granted Patent B2
US 10,737,702 · App. 15/221,002 · Granted Aug 11, 2020

Visually simulating driving plans in autonomous vehicles

Inventor: John-Michael McNew (Ann Arbor, MI)
Assignee: Toyota Motor Engineering & Manufacturing North America, Inc.
B60W50/14B60W30/08B60W50/0097G08G1/0962B60W2050/146B60W2540/215B60W2554/00B60W2556/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,737,702
App. No.
15/221,002
Granted
Aug 11, 2020
Kind
B2
Abstract

A method of autonomous driving includes evaluating information about an environment surrounding a vehicle, identifying a driving maneuver, and generating a driving plan for performing the driving maneuver based on the evaluation of the information about the environment surrounding the vehicle. Additionally, to apprise a user of a risk of performing the driving maneuver, the method includes visually simulating the driving plan by displaying, on a display, a sequence of actions associated with performing the driving maneuver. Further, the method includes outputting, at at least one interface, a query whether the user confirms the driving maneuver. When, in response to the query, a user response is received, at the at least one interface, that the user confirms the driving maneuver, the method includes operating vehicle systems in the vehicle to perform the driving maneuver according to the driving plan.

Claims (68)

1. A method of autonomous driving, comprising:

evaluating, using a perception module executable by at least one processor, information about an environment surrounding a vehicle;

using a planning/decision making module executable by the at least one processor:

identify a driving maneuver from a user request for the vehicle to perform the driving maneuver received at at least one interface;

generate a driving plan for performing the driving maneuver based on the evaluation of the information about the environment surrounding the vehicle;

conditionally upon categorizing the driving maneuver as feasible, but with some risk, to apprise a user of the risk of performing the driving maneuver, visually simulate the driving plan by displaying, on a display, a sequence of actions associated with performing the driving maneuver, thereby conveying at least one problem with performing the driving maneuver with which the risk of performing the driving maneuver is associated; and

output, at the at least one interface, a query whether the user confirms the driving maneuver; and

operating, using a control module executable by the at least one processor, vehicle systems in the vehicle to perform the driving maneuver according to the driving plan conditionally upon receiving a user response to the query, at the at least one interface, that the user confirms the driving maneuver.

2. The method of claim 1 , wherein visually simulating the driving plan includes displaying, on the display, a sequence of relationships between the displayed actions associated with performing the driving maneuver and the environment surrounding the vehicle.

3. The method of claim 1 , wherein visually simulating the driving plan includes displaying, on the display, a sequence of relationships between the displayed actions associated with performing the driving maneuver, multiple lane positions of a roadway along which the vehicle is maneuvering, and a predicted future maneuvering of at least one obstacle along the roadway.

4. The method of claim 1 , wherein the displayed actions associated with performing the driving maneuver include at least one of acceleration, braking, steering and signaling associated with performing the driving maneuver.

5. The method of claim 1 , further comprising:

using the planning/decision making module executable by the at least one processor:

determine the risk of performing the driving maneuver; and

categorize the driving maneuver based on the risk of its performance.

6. The method of claim 1 , further comprising:

using the planning/decision making module executable by the at least one processor:

identify the at least one problem with performing the driving maneuver, the at least one problem with performing the driving maneuver including at least one of a comfort problem and a navigational problem; and

determine the risk of performing the driving maneuver based on the at least one problem with performing the driving maneuver; wherein

the conveyed at least one problem with performing the driving maneuver includes the at least one of a comfort problem and a navigational problem.

7. A vehicle, comprising:

sensors configured to detect information about an environment surrounding the vehicle;

a display;

at least one interface;

vehicle systems operable to maneuver the vehicle; and

one or more modules stored on memory and executable by at least one processor for initiating instructions, the instructions including:

evaluating information about the environment surrounding the vehicle detected by the sensors;

identifying a driving maneuver from a user request for the vehicle to perform the driving maneuver received at the at least one interface;

generating a driving plan for performing the driving maneuver based on the evaluation of the information about the environment surrounding the vehicle;

conditionally upon categorizing the driving maneuver as feasible, but with some risk, to apprise a user of the risk of performing the driving maneuver, visually simulating the driving plan by displaying, on the display, a sequence of actions associated with performing the driving maneuver, thereby conveying at least one problem with performing the driving maneuver with which the risk of performing the driving maneuver is associated;

outputting, at the at least one interface, a query whether the user confirms the driving maneuver; and

operating the vehicle systems to perform the driving maneuver according to the driving plan conditionally upon receiving a user response to the query, at the at least one interface, that the user confirms the driving maneuver.

8. The vehicle of claim 7 , wherein visually simulating the driving plan includes displaying, on the display, a sequence of relationships between the displayed actions associated with performing the driving maneuver and the environment surrounding the vehicle.

9. The vehicle of claim 7 , wherein visually simulating the driving plan includes displaying, on the display, a sequence of relationships between the displayed actions associated with performing the driving maneuver, multiple lane positions of a roadway along which the vehicle is maneuvering, and a predicted future maneuvering of at least one obstacle along the roadway.

10. The vehicle of claim 7 , wherein the displayed actions associated with performing the driving maneuver include at least one of acceleration, braking, steering and signaling associated with performing the driving maneuver.

11. The vehicle of claim 7 , wherein the instructions further include:

determining the risk of performing the driving maneuver; and

categorizing the driving maneuver based on the risk of its performance.

12. The vehicle of claim 7 , wherein the instructions further include:

identifying the at least one problem with performing the driving maneuver, the at least one problem with performing the driving maneuver including at least one of a comfort problem and a navigational problem; and

determining the risk of performing the driving maneuver based on the at least one problem with performing the driving maneuver; wherein

the conveyed at least one problem with performing the driving maneuver includes the at least one of a comfort problem and a navigational problem.

13. A vehicle, comprising:

sensors configured to detect information about an environment surrounding the vehicle;

a display;

at least one interface;

vehicle systems operable to maneuver the vehicle; and

one or more modules stored on memory and executable by at least one processor for initiating instructions, the instructions including:

evaluating information about the environment surrounding the vehicle detected by the sensors;

identifying a driving maneuver from a user request for the vehicle to perform the driving maneuver received at the at least one interface;

generating a driving plan for performing the driving maneuver based on the evaluation of the information about the environment surrounding the vehicle;

categorizing the driving maneuver, based on a risk of its performance, as infeasible, or as feasible, but with some risk;

when the driving maneuver is categorized as infeasible, rejecting the driving maneuver;

when the driving maneuver is categorized as feasible, but with some risk, provisionally accepting the driving maneuver;

conditionally upon provisionally accepting the driving maneuver, to apprise a user of the risk of performing the driving maneuver, visually simulating the driving plan by displaying, on the display, a sequence of actions associated with performing the driving maneuver, thereby conveying at least one problem with performing the driving maneuver with which the risk of performing the driving maneuver is associated;

outputting, at the at least one interface, a query whether the user confirms the driving maneuver;

non-provisionally accepting the driving maneuver conditionally upon receiving a user response to the query, at the at least one interface, that the user confirms the driving maneuver; and

when the driving maneuver is non-provisionally accepted, operating the vehicle systems to perform the driving maneuver according to the driving plan.

14. The vehicle of claim 13 , wherein the instructions further include:

categorizing the driving maneuver, based on a risk of its performance, as infeasible, as feasible, and substantially without risk, or as feasible, but with some risk; and

when the driving maneuver is categorized as feasible, and substantially without risk, non-provisionally accepting the driving maneuver.

15. The vehicle of claim 13 , wherein visually simulating the driving plan includes displaying, on the display, a sequence of relationships between the displayed actions associated with performing the driving maneuver and the environment surrounding the vehicle.

16. The vehicle of claim 13 , wherein visually simulating the driving plan includes displaying, on the display, a sequence of relationships between the displayed actions associated with performing the driving maneuver, multiple lane positions of a roadway along which the vehicle is maneuvering, and a predicted future maneuvering of at least one obstacle along the roadway.

17. The vehicle of claim 13 , wherein the displayed actions associated with performing the driving maneuver include at least one of acceleration, braking, steering and signaling associated with performing the driving maneuver.

18. The vehicle of claim 13 , wherein the instructions further include:

identifying the at least one problem with performing the driving maneuver, the at least one problem with performing the driving maneuver including at least one of a comfort problem and a navigational problem; and

determining the risk of performing the driving maneuver based on the at least one problem with performing the driving maneuver; wherein

the conveyed at least one problem with performing the driving maneuver includes the at least one of a comfort problem and a navigational problem.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 2, 2020
From: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 053673/0486 →
CHANGE OF ADDRESS Recorded Nov 30, 2018
From: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
To: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
Reel/Frame 047688/0784 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 2, 2016
From: MCNEW, JOHN-MICHAEL
To: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
Reel/Frame 039314/0919 →
Continuity (1)
Related Publication 20180029610A1 · Feb 1, 2018