IP Library › Granted Patent US 11,270,458
Granted Patent B2
US 11,270,458 · App. 16/690,946 · Granted Mar 8, 2022

Image generating device

Inventor: Koji Nishiyama (Nishinomiya, JP)
Assignee: Furuno Electric Co., Ltd.
G06T7/73G06T19/006H04N5/23293H04N13/366
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,270,458
App. No.
16/690,946
Granted
Mar 8, 2022
Kind
B2
Abstract

The present disclosure provides an image generating device that includes processing circuitry configured to acquire an image captured by an imaging device to be installed in a water-surface movable body, acquire positional information indicative of a position of the water-surface movable body, acquire posture information indicative of a posture of the water-surface movable body, acquire additional display information including information indicative of positions of one or more locations, generate a synthesized image where a graphic rendering a three-dimensional virtual reality object indicative of the additional display information is synthesized on the captured image based on the positional information, the posture information, and the additional display information, and place the graphic across a boundary of the captured image when the captured image is placed only in a certain portion of the synthesized image.

Claims (53)

1. An image generating device, comprising:

processing circuitry configured to:

acquire an image captured by an imaging device to be installed in a water-surface movable body;

acquire positional information indicative of a position of the water-surface movable body;

acquire posture information indicative of a posture of the water-surface movable body;

acquire additional display information including information indicative of positions of one or more locations;

generate a synthesized image where a graphic rendering a three-dimensional virtual reality object indicative of the additional display information is synthesized on the captured image based on the positional information, the posture information, and the additional display information; and

place the graphic across a boundary of the captured image when the captured image is placed only in a certain portion of the synthesized image.

2. The image generating device of claim 1 , wherein the processing circuitry is further configured to synthesize the graphic rendering the virtual reality object having the shape of a ship with the captured image.

3. The image generating device of claim 2 , wherein the processing circuitry is further configured to synthesize with the captured image the graphic rendering the virtual reality object where the additional display information is based on at least any of a length, a width, a bow direction of the ship.

4. The image generating device of claim 2 , wherein the processing circuitry is further configured to:

synthesize a directional scale indicative of an azimuth on the captured image; and

place the directional scale across the captured image when the captured image is placed only in the certain portion of the synthesized image.

5. The image generating device of claim 4 , wherein the processing circuitry is further configured to:

synthesize a horizon that is a boundary between a water surface and sky with the captured image; and

place the horizon across the boundary of the captured image when the captured image is placed only in the certain portion of the synthesized image.

6. The image generating device of claim 5 , wherein the processing circuitry is further configured to:

synthesize a grid indicative of a water surface with the captured image; and

place the grid across the boundary of the captured image when the captured image is placed only in the certain portion of the synthesized image.

7. The image generating device of claim 6 , wherein the processing circuitry is further configured to render the boundary of the captured image obscured or emphasized.

8. The image generating device of claim 7 , wherein the processing circuitry is further configured to:

perform, when the captured image is placed in the synthesized image, a posture change tracking in which a rendering position and direction of the virtual reality object are changed corresponding at least to a posture change of an imaging optical axis of the imaging device in the roll direction.

9. The image generating device of claim 8 , wherein the processing circuitry is further configured to:

stop the posture change tracking when the captured image is not placed at all in the synthesized image.

10. The image generating device of claim 1 , wherein the processing circuitry is further configured to:

synthesize a directional scale indicative of an azimuth on the captured image; and

place the directional scale across the captured image when the captured image is placed only in the certain portion of the synthesized image.

11. The image generating device of claim 1 , wherein the processing circuitry is further configured to:

synthesize a horizon that is a boundary between a water surface and sky with the captured image; and

place the horizon across the boundary of the captured image when the captured image is placed only in the certain portion of the synthesized image.

12. The image generating device of claim 1 , wherein the processing circuitry is further configured to:

synthesize a grid indicative of a water surface with the captured image; and

place the grid across the boundary of the captured image when the captured image is placed only in the certain portion of the synthesized image.

13. The image generating device of claim 1 , wherein the processing circuitry is further configured to render the boundary of the captured image obscured or emphasized.

14. The image generating device of claim 1 , wherein the processing circuitry is further configured to:

perform, when the captured image is placed in the synthesized image, a posture change tracking in which a rendering position and direction of the virtual reality object are changed corresponding at least to a posture change of an imaging optical axis of the imaging device in the roll direction.

15. The image generating device of claim 1 , wherein the processing circuitry is further configured to:

stop a posture change tracking when the captured image is not placed at all in the synthesized image.

16. The image generating device of claim 1 , wherein the processing circuitry is further configured to:

place at least one of the additional display information as the three-dimensional virtual reality object in a three-dimensional virtual space;

place the captured image based on the positional information and the posture information;

generate the synthesized image by rendering the virtual reality object and the captured image based on the position and the direction of a virtual camera located in the three-dimensional virtual space; and

change at least any of the position and the direction of the virtual camera based on an instruction of a user.

17. The image generating device of claim 16 , wherein the processing circuitry is further configured to automatically change at least any of the position and the direction of the virtual camera so that a point in the three-dimensional virtual space, or when the user selects the virtual reality object, the selected point or the selected virtual reality object is continuously included in the synthesized image.

18. The image generating device of claim 1 , wherein the processing circuitry is further configured to generate a scale image indicative of an azimuth; and

determine a display position of the scale image according to the position where the graphic is displayed.

19. A method of generating an image, comprising:

acquiring an image captured by an imaging device to be installed in a water-surface movable body;

acquiring positional information indicative of a position of the water-surface movable body;

acquiring posture information indicative of a posture of the water-surface movable body;

acquiring additional display information including information indicative of positions of one or more locations;

generating a synthesized image where a graphic rendering a three-dimensional virtual reality object indicative of the additional display information is synthesized on the captured image based on the positional information, the posture information, and the additional display information; and

placing the graphic across a boundary of the captured image when the captured image is placed only in a certain portion of the synthesized image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 21, 2019
From: NISHIYAMA, KOJI
To: FURUNO ELECTRIC CO., LTD.
Reel/Frame 051078/0849 →
Priority Claims (1)
JP JP2017-103150 · May 24, 2017 · national
Continuity (2)
Continuation In Part PCTJP2018018648 · May 15, 2018
Related Publication 20200090367A1 · Mar 19, 2020
Cited By (1)
US 12,188,766