IP Library Granted Patent US 11,535,155
Granted Patent B2
US 11,535,155 · App. 16/649,365 · Granted Dec 27, 2022

Superimposed-image display device and computer program

Inventors: Kenji Watanabe (Okazaki, JP); Hiroyuki Miyake (Toyota, JP); Yuusuke Morita (Okazaki, JP)
Assignee: AISIN CORPORATION
B60R1/001B60K35/00B60K2370/149B60K2370/1529B60K2370/166B60K2370/785B60R2300/205B60R2300/308
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,535,155
App. No.
16/649,365
Granted
Dec 27, 2022
Kind
B2
Abstract

Superimposed-image display devices and programs display a guide image providing a guidance of information to a driver of a vehicle such that the guide image is superimposed on a view ahead of the vehicle and is visually recognized. The systems and programs obtain three-dimensional map information that specifies a three-dimensional shape of a road and a structure nearby the road and arrange the guide image in the three-dimensional map information, based on the three-dimensional shape of the road and the structure nearby the road. The systems and programs obtain a shape of the guide image that is visually recognized from a position of the driver in the three-dimensional map information and display the guide image having the obtained shape.

Claims (33)

1. A superimposed image display device that is mounted on a vehicle and that displays a guide image such that the guide image is superimposed on a view ahead of the vehicle and is visually recognized, the guide image providing guidance of information to a driver of the vehicle, the superimposed image display device comprising:

a processor programmed to:

obtain three-dimensional map information that specifies a three-dimensional shape of a road and a structure nearby the road;

obtain a captured image that captures a view ahead of the vehicle;

compare the captured image and the three-dimensional map information at prescribed intervals, the prescribed interval is an interval where a comparison is done each time a structure having a distinctive shape, which can be easily compared, is included in the captured image;

specify a position of the driver in the three-dimensional map information, based on a result of the comparison;

estimate the position of the driver in the three-dimensional map information, during a time between timings at which the comparison is performed;

specify as the position of the driver in the three-dimensional map information, the estimated position of the driver during the time between timings at which the comparison is performed;

arrange the guide image in the three-dimensional map information, based on the three-dimensional shape of the road and the structure nearby the road;

obtain a shape of the guide image that is visually recognized from the position of the driver in the three-dimensional map information; and

display the guide image having the obtained shape.

2. The superimposed image display device according to claim 1 , wherein the guide image has a two-dimensional shape or a three-dimensional shape.

3. The superimposed image display device according to claim 1 , wherein:

the position of the driver is a position of an eye of the driver; and

the processor is programmed to:

obtain the position of the eye of the driver, and

obtain the shape of the guide image that is visually recognized from the position of the eye of the driver in the three-dimensional map information.

4. The superimposed image display device according to claim 1 , wherein the guide image includes at least one of an image indicating a route along which the vehicle travels, a sign image, and a guidance line image on a road surface.

5. The superimposed image display device according to claim 4 , wherein the guidance line image on the road surface is a road surface marking image.

6. The superimposed image display device according to claim 4 , wherein the guidance line image on the road surface is superimposed with a boundary between a roadway and a sidewalk or an edge of a roadway.

7. The superimposed image display device according to claim 1 , wherein the structure is at least one building or road sign.

8. The superimposed image display device according to claim 1 , wherein a virtual object that specifies a three-dimensional shape of a moving object nearby the vehicle is arranged in the three-dimensional map information.

9. A computer-readable storage medium storing a computer-executable computer program causing a computer to perform functions, comprising:

displaying a guide image providing a guidance of information to a driver of a vehicle such that the guide image is superimposed on a view ahead of the vehicle and is visually recognized;

obtaining three-dimensional map information that specifies a three-dimensional shape of a road and a structure nearby the road;

obtaining a captured image that captures a view ahead of the vehicle;

comparing the captured image and the three-dimensional map information at prescribed intervals, the prescribed interval is an interval where a comparison is done each time a structure having a distinctive shape, which can be easily compared, is included in the captured image;

specifying a position of the driver in the three-dimensional map information, based on a result of the comparison;

estimating the position of the driver in the three-dimensional map information, during a time between timings at which the comparison is performed;

specifying as the position of the driver in the three-dimensional map information, the estimated position of the driver during the time between timings at which the comparison is performed;

arranging the guide image in the three-dimensional map information, based on the three-dimensional shape of the road and the structure nearby the road;

obtaining a shape of the guide image that is visually recognized from the position of the driver in the three-dimensional map information; and

displaying the guide image having the obtained shape.

Assignments (2)
MERGER Recorded Dec 13, 2021
From: AISIN AW CO., LTD.
To: AISIN CORPORATION
Reel/Frame 058487/0162 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 20, 2020
From: WATANABE, KENJI; MIYAKE, HIROYUKI; MORITA, YUUSUKE
To: AISIN AW CO., LTD.
Reel/Frame 052203/0326 →