IP Library › Granted Patent US 10,048,080
Granted Patent B2
US 10,048,080 · App. 15/077,225 · Granted Aug 14, 2018

Autonomous vehicle virtual reality navigation system

Inventors: Kristofer D. Kusano (Ann Arbor, MI); Yi Li (Ann Arbor, MI); Hideki Hada (Ann Arbor, MI); Miles J. Johnson (Ann Arbor, MI)
Assignee: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
G01C21/3415G01C21/365G01C21/3667
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,048,080
App. No.
15/077,225
Granted
Aug 14, 2018
Kind
B2
Abstract

The virtual reality navigation system can display the position of an autonomous vehicle at a predetermined amount of time in the future via virtual reality. A virtual reality display can display a virtual reality autonomous vehicle, as if the virtual reality autonomous vehicle is the autonomous vehicle operating at the predetermined amount of time in the future. Additionally, a virtual reality interface can provide an interface for an operator to interact with the environment of the autonomous vehicle. The virtual reality interface can receive an alternate route selection, and display in virtual reality the one or more maneuvers that would be required to reach the selection.

Claims (57)

1. An autonomous vehicle system comprising:

a plurality of sensors;

a virtual reality display;

a virtual reality interface; and

circuitry configured to

receive output from the plurality of sensors,

receive a current position of the vehicle via a positioning system,

display the vehicle via the virtual reality display at a first predetermined amount of time in the future based on the output from the plurality of sensors and the current position of the vehicle,

receive a selection for a first alternate route, the first alternate route being a preferred alternate route based on the selection,

determine if the first alternate route is available,

display the vehicle executing one or more maneuvers to reach the first alternate route when the first alternate route is available, and

display the vehicle on the first alternate route in a position corresponding to the first predetermined amount of time in the future when the first alternate route is available.

2. The autonomous vehicle system of claim 1 , wherein the circuitry is further configured to

automatically suggest a second alternate route when the first alternate route is not available, the second alternate route being an available alternate route most similar to the first alternate route,

determine if the second alternate route is selected within a second predetermined amount of time,

display the vehicle executing one or more maneuvers to reach the second alternate route when the second alternate route is selected within the predetermined amount of time, and

display the vehicle on the second alternate route in a position corresponding to the first predetermined amount of time in the future when the second alternate route is selected within the second predetermined amount of time.

3. The autonomous vehicle system of claim 2 , wherein the first predetermined amount of time is indicative of the position of the vehicle that amount of time in the future, the position of the vehicle being a predetermined position based on a planned route of the autonomous vehicle.

4. The autonomous vehicle system of claim 3 , wherein the first predetermined amount of time can be adjusted via the virtual reality interface.

5. The autonomous vehicle system of claim 1 , wherein the virtual reality display displays a 2D animation of the vehicle and its surroundings operating at the first predetermined amount of time in the future.

6. The autonomous vehicle system of claim 1 , wherein the virtual reality display displays a 3D animation of the vehicle and its surroundings operating at the first predetermined amount of time in the future.

7. The autonomous vehicle system of claim 6 , wherein the virtual reality display is a display system including a Heads Up Display (HUD), a hologram projection system, or a liquid crystal display (LCD).

8. The autonomous vehicle system of claim 2 , wherein the selection for the first alternate route is an interaction via the virtual reality interface.

9. The autonomous vehicle system of claim 2 , wherein the automatic suggestion for the second alternate route is displayed on the virtual reality interface.

10. The autonomous vehicle system of claim 9 , wherein the second alternate route is confirmed within the second predetermined amount of time via an interaction with the virtual reality interface.

11. The autonomous vehicle system of claim 9 , wherein the second alternate route is automatically suggested when the autonomous vehicle is not able to safely execute one or more maneuvers to reach the first alternate route.

12. The autonomous vehicle system of claim 2 , wherein the circuitry is configured to not execute a maneuver to any alternate route when the second alternate route is not selected within the predetermined amount of time.

13. The autonomous vehicle system of claim 2 , wherein the one or more maneuvers executed by the vehicle to reach any alternate route are displayed as an animation on the virtual reality display.

14. The autonomous vehicle system of claim 2 , wherein the one or more maneuvers executed by the vehicle to reach any alternate route are displayed as an animation on the virtual reality interface.

15. The autonomous vehicle system of claim 2 , wherein the one or more maneuvers executed by the vehicle to reach any alternate route are displayed as an animation on the virtual reality display and the virtual reality interface.

16. The autonomous vehicle of system claim 15 , wherein the one or more maneuvers executed by the vehicle to reach any alternate route are displayed up until the first predetermined amount of time in the future.

17. A method of displaying a virtual reality navigation system comprising:

receiving output from a plurality of sensors;

receiving a current position of a vehicle via a positioning system;

displaying the vehicle via the virtual reality display at a first predetermined amount of time in the future based on the output from the plurality of sensors and the current position of the vehicle;

receiving a selection for a first alternate route, the first alternate route being a preferred alternate route based on the selection;

determining, via processing circuitry, if the first alternate route is available;

displaying the vehicle executing one or more maneuvers to reach the first alternate route when the first alternate route is available; and

displaying the vehicle on the first alternate route in a position corresponding to the first predetermined amount of time in the future when the first alternate route is available.

18. The method of claim 17 , further comprising:

automatically suggesting a second alternate route when the first alternate route is not available, the second alternate route being an available alternate route most similar to the first alternate route;

determining, via processing circuitry, if the second alternate route is selected within a second predetermined amount of time;

displaying the vehicle executing one or more maneuvers to reach the second alternate route when the second alternate route is selected within the predetermined amount of time; and

displaying the vehicle on the second alternate route in a position corresponding to the first predetermined amount of time in the future when the second alternate route is selected within the second predetermined amount of time.

19. A non-transitory computer-readable storage medium storing computer-readable instructions that, when executed by a computer, cause the computer to perform the method of:

receiving output from a plurality of sensors;

receiving a current position of a vehicle via a positioning system;

displaying the vehicle via the virtual reality display at a first predetermined amount of time in the future based on the output from the plurality of sensors and the current position of the vehicle;

receiving a selection for a first alternate route, the first alternate route being a preferred alternate route based on the selection;

determining if the first alternate route is available;

displaying the vehicle executing one or more maneuvers to reach the first alternate route when the first alternate route is available; and

displaying the vehicle on the first alternate route in a position corresponding to the first predetermined amount of time in the future when the first alternate route is available.

20. The method of claim 19 , further comprising:

automatically suggesting a second alternate route when the first alternate route is not available, the second alternate route being an available alternate route most similar to the first alternate route;

determining if the second alternate route is selected within a second predetermined amount of time,

displaying the vehicle executing one or more maneuvers to reach the second alternate route when the second alternate route is selected within the predetermined amount of time; and

displaying the vehicle on the second alternate route in a position corresponding to the first predetermined amount of time in the future when the second alternate route is selected within the second predetermined amount of time.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 23, 2018
From: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 046678/0322 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 22, 2016
From: KUSANO, KRISTOFER D.; LI, YI; HADA, HIDEKI; JOHNSON, MILES J.
To: TOYOTA MOTOR ENGINEERING & MANUFACTURING NORTH AMERICA, INC.
Reel/Frame 038067/0422 →
Continuity (1)
Related Publication 20170276494A1 · Sep 28, 2017