IP Library Granted Patent US 11,302,076
Granted Patent B2
US 11,302,076 · App. 16/338,886 · Granted Apr 12, 2022

Periphery monitoring apparatus

Inventor: Kazuya Watanabe (Anjo, JP)
Assignee: AISIN CORPORATION
G06T19/003B60R1/002G06T7/70B60R2300/602G06T2207/30252G06T2219/028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,302,076
App. No.
16/338,886
Granted
Apr 12, 2022
Kind
B2
Abstract

A periphery monitoring apparatus includes: an image capturing unit configured to generate a captured image by capturing a periphery of a mobile body; and a controller configured to generate, from the captured image, a peripheral image representing a view from a virtual viewpoint that is set at a relative position to a current position of the mobile body depending on a movement of the mobile body, generate a display image in which a mobile body image representing a position of the mobile body is superimposed on the peripheral image, and display the mobile body image to fix a position of the mobile body image in the display image.

Claims (44)

1. A periphery monitoring apparatus comprising:

an image capturing unit configured to generate a captured image by capturing a periphery of a mobile body; and

a processor configured to implement:

a controller configured to

generate, from the captured image, a peripheral image representing a view from a virtual viewpoint that is set at a relative position to a current position of the mobile body depending on a movement of the mobile body,

generate a display image in which a mobile body image representing a position of the mobile body is superimposed on the peripheral image

display the display image on a display device, and

update the display image according to a moving distance of the mobile body,

wherein that the controller is further configured to

carry out the generation of the peripheral image by projecting a past captured image onto a preset virtual projection plane, wherein the past captured image has been captured by the image capturing unit in the past at an imaged position that is different from the current position,

carry out the update of the display image by:

generating a new peripheral image based on a new current position of the mobile body, and

generating an updated display image by superimposing the mobile body image of the display image onto the new peripheral image without changing a position of the mobile body image within a screen of the display device.

2. The periphery monitoring apparatus according to claim 1 , wherein the controller generates, as the peripheral image, an image representing a view of the captured image from the virtual point, the captured image being projected onto a three-dimensional virtual projection plane including a first virtual plane and a second virtual plane intersecting with the first virtual plane.

3. The periphery monitoring apparatus according to claim 1 , wherein the controller moves the virtual viewpoint in accordance with an input from a user.

4. The periphery monitoring apparatus according to claim 1 , further comprising a storage unit configured to store a three-dimensional mobile body shape model that is a three-dimensional shape model of the mobile body,

wherein the mobile body image is generated from the three-dimensional mobile body shape model stored in advance in the storage unit.

5. The periphery monitoring apparatus according to claim 1 , wherein the controller generates, as the peripheral image, an image representing a view of the captured image from the virtual point, the captured image being projected onto a three-dimensional virtual projection plane including a first two-dimensional virtual plane and a second two-dimensional virtual plane intersecting with the first virtual plane.

6. A periphery monitoring method comprising:

generating a captured image by capturing a periphery of a mobile body;

generating, from the captured image, a peripheral image representing a view from a virtual viewpoint that is set at a relative position to a current position of the mobile body depending on a movement of the mobile body;

generating a display image in which a mobile body image representing a position of the mobile body is superimposed on the peripheral image;

displaying the display image on a display device,

update the display image according to a moving distance of the mobile body,

carry out the generation of the peripheral image by projecting a past captured image onto a preset virtual projection plane, wherein the past captured image has been captured by the image capturing unit in the past at an imaged position that is different from the current position,

carry out the update of the display image by:

generating a new peripheral image based on a new current position of the mobile body, and

generating an updated display image by superimposing the mobile body image of the display image onto the new peripheral image without changing a position of the mobile body image within a screen of the display device.

7. The periphery monitoring method according to claim 6 , wherein the generating a peripheral image includes generating, as the peripheral image, an image representing a view of the captured image from the virtual point, the captured image being projected onto a three-dimensional virtual projection plane including a first virtual plane and a second virtual plane intersecting with the first virtual plane.

8. The periphery monitoring method according to claim 6 , further comprising moving the virtual viewpoint in accordance with an input from a user.

9. The periphery monitoring method according to claim 6 , further comprising storing, in a memory, a three-dimensional mobile body shape model that is a three-dimensional shape model of the mobile body,

wherein the mobile body image is generated from the three-dimensional mobile body shape model stored in advance in the memory.

10. The periphery monitoring method according to claim 6 , wherein the generating a peripheral image includes generating, as the peripheral image, an image representing a view of the captured image from the virtual point, the captured image being projected onto a three-dimensional virtual projection plane including a first two-dimensional virtual plane and a second two-dimensional virtual plane intersecting with the first virtual plane.

11. A non-transitory computer readable recording medium on which an executable program is recorded, the program comprising instructions which cause a computer to execute:

generating a captured image by capturing a periphery of a mobile body;

generating, from the captured image, a peripheral image representing a view from a virtual viewpoint that is set at a relative position to a current position of the mobile body depending on a movement of the mobile body;

generating a display image in which a mobile body image representing a position of the mobile body is superimposed on the peripheral image;

displaying the display image on a display device, and

update the display image according to a moving distance of the mobile body,

wherein that the controller is further configured to

carry out the generation of the peripheral image by projecting a past captured image onto a preset virtual projection plane, wherein the past captured image has been captured by the image capturing unit in the past at an imaged position that is different from the current position, carry out the update of the display image by:

generating a new peripheral image based on a new current position of the mobile body, and

generating an updated display image by superimposing the mobile body image of the display image onto the new peripheral image without changing a position of the mobile body image within a screen of the display device.

12. The non-transitory computer readable recording medium according to claim 11 , wherein the generating a peripheral image includes generating, as the peripheral image, an image representing a view of the captured image from the virtual point, the captured image being projected onto a three-dimensional virtual projection plane including a first two-dimensional virtual plane and a second two-dimensional virtual plane intersecting with the first virtual plane.

Assignments (2)
CHANGE OF NAME Recorded Dec 23, 2021
From: AISIN SEIKI KABUSHIKI KAISHA
To: AISIN CORPORATION
Reel/Frame 058575/0964 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 2, 2019
From: WATANABE, KAZUYA
To: AISIN SEIKI KABUSHIKI KAISHA
Reel/Frame 048772/0247 →
Priority Claims (1)
JP JP2016-196357 · Oct 4, 2016 · national
Continuity (1)
Related Publication 20210279959A1 · Sep 9, 2021
Cited By (1)
US 12,475,620