IP Library Granted Patent US 11,328,593
Granted Patent B2
US 11,328,593 · App. 16/527,495 · Granted May 10, 2022

Autonomous vehicle user interface with predicted trajectories

Inventors: Hiromitsu Urano (Ann Arbor, MI); Maiko Hirano (Nagoya, JP); Yusuke Sawamura (Ann Arbor, MI)
Assignee: Toyota Research Institute, Inc.
G08G1/096855G01C21/365G05D1/0214G08G1/0125G08G1/166G05D2201/0213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,328,593
App. No.
16/527,495
Granted
May 10, 2022
Kind
B2
Abstract

Systems and methods are provided for generating trajectories for a vehicle user interface showing a driver's perspective view. Methods include generating an ego-vehicle predicted trajectory for an ego-vehicle; and generating at least one road agent predicted trajectory for a road agent that is external to the ego-vehicle. After the predicted trajectories are generated, the method continues by determining that at least two predicted trajectories overlap when displayed on the user interface showing a driver's perspective view, when displayed on the user interface showing a driver's perspective view. The method includes modifying the at least one road agent predicted trajectory to remove the overlap. The method then proceeds with updating a display of the user interface to include any modified road agent predicted trajectory. Systems include a trajectory-prediction module to execute the methods.

Claims (79)

1. A method for modifying trajectories displayed on a vehicle user interface showing a driver's perspective view, the method comprising:

generating an ego-vehicle predicted trajectory for an ego-vehicle;

generating at least one road agent predicted trajectory for a road agent that is external to the ego-vehicle;

determining that at least two predicted trajectories overlap when displayed on the user interface showing a driver's perspective view;

controlling operation of the user interface to clarify a display of at least one predicted trajectory of the at least two overlapping predicted trajectories by modifying the display to remove the overlap from the display; and

controlling operation of the user interface to update the user interface to include any modified road agent predicted trajectory, wherein:

the road agent is one of an automobile, a motorcycle, a bicycle, and a pedestrian; and

the vehicle user interface is one of a navigation display, a multi-information display, a heads-up display (HUD), a head mounted display (HMD), a remote operator display, and a wearable device.

2. The system according to claim 1 , wherein the instruction to modify at least one predicted trajectory to remove the overlap comprises an instruction to:

modify a length of at least one of the predicted trajectories.

3. The system according to claim 1 , wherein the instruction to modify at least one predicted trajectory to remove the overlap comprises an instruction to:

provide a separation distance between the at least two predicted trajectories.

4. The method according to claim 1 , wherein modifying the display to remove the overlap from the display comprises at least one of:

modifying a length of at least one of the predicted trajectories; and

providing a separation distance between the at least two predicted trajectories.

5. A system for modifying displays of trajectories on a vehicle user interface showing a driver's perspective view, the system comprising:

one or more processors; and

a memory communicably coupled to the one or more processors and storing a trajectory-prediction module including computer-readable instructions that when executed by the one or more processors cause the one or more processors to:

generate an ego-vehicle predicted trajectory for an ego-vehicle;

generate at least one road agent predicted trajectory for a road agent that is external to the ego-vehicle;

determine that at least two predicted trajectories overlap when displayed on the user interface showing a driver's perspective view;

determine that a confidence score representative of a possibility of a collision occurring based on the overlap is less than a predetermined threshold;

determine that at least one road agent will arrive at a location of the overlap before the ego-vehicle; and

control operation of the user interface to shorten a length of a displayed image of at least one predicted trajectory to remove the overlap from the display,

the memory also storing a control module including computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to control operation of the user interface to update the user interface to display any modified road agent predicted trajectory.

6. A system for modifying displays of trajectories on a vehicle user interface showing a driver's perspective view, the system comprising:

one or more processors; and

a memory communicably coupled to the one or more processors and storing a trajectory-prediction module including computer-readable instructions that when executed by the one or more processors cause the one or more processors to:

generate an ego-vehicle predicted trajectory for an ego-vehicle;

generate at least one road agent predicted trajectory for a road agent that is external to the ego-vehicle;

determine that at least two predicted trajectories overlap at a first intersection when displayed on the user interface showing a driver's perspective view;

determine that at least two road agent predicted trajectories overlap at a second intersection;

determine a priority intersection based on a calculation of which road agents will first arrive at one of the first and the second intersections; and

control operation of the user interface to modify at least one road agent predicted trajectory that overlaps at the priority intersection by removing the overlap from the display,

the memory also storing a control module including computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to control operation of the user interface to update the user interface to display any modified road agent predicted trajectory.

7. A system for modifying displays of trajectories on a vehicle user interface showing a driver's perspective view, the system comprising:

one or more processors; and

a memory communicably coupled to the one or more processors and storing a trajectory-prediction module including computer-readable instructions that when executed by the one or more processors cause the one or more processors to:

generate an ego-vehicle predicted trajectory for an ego-vehicle;

generate at least one road agent predicted trajectory for a road agent that is external to the ego-vehicle;

determine that a first road agent predicted trajectory overlaps with a first object at a first intersection when displayed on the user interface showing a driver's perspective view;

determine that a second road agent predicted trajectory overlaps with one of a second object and a third road agent predicted trajectory at a second intersection;

determine a priority intersection based on a calculation of which road agents will first arrive at one of the first and the second intersections; and

control operation of the user interface to modify the display of at least one road agent predicted trajectory that overlaps at the priority intersection by removing the overlap from the display,

the memory also storing a control module including computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to control operation of the user interface to update the user interface to display any modified road agent predicted trajectory.

8. A system for modifying displays of trajectories on a vehicle user interface showing a driver's perspective view, the system comprising:

one or more processors; and

a memory communicably coupled to the one or more processors and storing a trajectory-prediction module including computer-readable instructions that when executed by the one or more processors cause the one or more processors to:

generate an ego-vehicle predicted trajectory for an ego-vehicle;

generate at least one road agent predicted trajectory for a road agent that is external to the ego-vehicle;

determine that at least two predicted trajectories overlap when displayed on the user interface showing a driver's perspective view; and

control operation of the user interface to clarify a display of at least one predicted trajectory by performing at least one of:

changing a color of at least one of the road agent predicted trajectories;

changing a spacing location of at least one of the road agent predicted trajectories;

changing a thickness of at least one of the road agent predicted trajectories; and

determining a priority road agent predicted trajectory based on a closest proximity to the ego-vehicle, and only displaying the priority road agent predicted trajectory,

the memory also storing a control module including computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to control operation of the user interface to update the user interface to display any modified road agent predicted trajectory.

9. The system according to claim 8 , wherein the trajectory-prediction module includes further instructions to:

control operation of the user interface to modify the display of the at least one predicted trajectory using at least one technique selected from the group consisting of hiding, diluting, and blending.

10. A system for modifying displays of trajectories on a vehicle user interface showing a driver's perspective view, the system comprising:

one or more processors; and

a memory communicably coupled to the one or more processors and storing:

a trajectory-prediction module including computer-readable instructions that when executed by the one or more processors cause the one or more processors to:

generate an ego-vehicle predicted trajectory for an ego-vehicle;

generate at least one road agent predicted trajectory for a road agent that is external to the ego-vehicle;

determine that a distance between two adjacent predicted trajectories is below a predetermined threshold value when displayed on the user interface showing a driver's perspective view; and

control operation of the user interface to clarify a display of at least one of the road agent predicted trajectories by performing at least one modification of the display, the at least one modification being selected from the group consisting of:

change a color of at least one of the road agent predicted trajectories;

change a spacing location of at least one of the road agent predicted trajectories;

change a thickness of at least one of the road agent predicted trajectories; and

determine a priority road agent predicted trajectory based on a closest proximity to the ego-vehicle, and only display the priority road agent predicted trajectory; and

a control module including computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to update the display of the user interface to include any modified road agent predicted trajectory.

11. A method for modifying trajectories displayed on a vehicle user interface showing a driver's perspective view, the method comprising:

generating an ego-vehicle predicted trajectory for an ego-vehicle;

generating at least one road agent predicted trajectory for a road agent that is external to the ego-vehicle;

determining that at least two predicted trajectories overlap when displayed on the user interface showing a driver's perspective view;

determining that a confidence score representative of a possibility of a collision occurring based on the overlap is less than a predetermined threshold;

controlling operation of the user interface to modify the display of the at least one predicted trajectory to remove the overlap from the display using at least one technique selected from the group consisting of hiding, diluting, and blending; and

controlling operation of the user interface to update the user interface to include any modified road agent predicted trajectory.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 10, 2022
From: TOYOTA RESEARCH INSTITUTE, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 060158/0058 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 14, 2019
From: URANO, HIROMITSU; HIRANO, MAIKO; SAWAMURA, YUSUKE
To: TOYOTA RESEARCH INSTITUTE, INC.
Reel/Frame 050045/0876 →
Continuity (1)
Related Publication 20210035447A1 · Feb 4, 2021