IP Library Granted Patent US 10,997,889
Granted Patent B2
US 10,997,889 · App. 16/605,269 · Granted May 4, 2021

Display control device, display control method, program, and display system

Inventors: Yoshiyuki Onishi (Tokyo, JP); Masahiro Kurihara (Tokyo, JP)
Assignee: KOMATSU LTD.
G09G3/20G06T19/00G09G5/377G09G5/38
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,997,889
App. No.
16/605,269
Granted
May 4, 2021
Kind
B2
Abstract

A position/pose acquisition unit acquires a position and a pose of a viewpoint in a real space. A site data disposition unit disposes site data which is three-dimensional data representing a shape of a construction site in a virtual space on the basis of the position and the pose of the viewpoint. A machine data disposition unit disposes machine data which is three-dimensional data representing a shape of a work machine at a position corresponding to the site data in the virtual space. A rendering unit renders the disposed machine data.

Claims (76)

1. A display control device comprising:

a position/pose acquisition unit configured to acquire a position and a pose of a first viewpoint in a real space according to a position and a pose of a portable terminal comprising a display device;

a site data disposition unit configured to dispose site data which is three-dimensional data representing a shape of a construction site in a virtual space on the basis of the position and the pose of the first viewpoint;

a machine data disposition unit configured to dispose machine data which is three-dimensional data representing a shape of a work machine at a position corresponding to the site data in the virtual space;

a rendering unit configured to render the disposed machine data according to a second viewpoint in the virtual space; and

a display control unit configured to output the rendered disposed machine data to the display device,

wherein the site data disposition unit disposes the site data such that a relative position and direction between the second viewpoint and the disposed site data correspond to a relative position and direction between the first viewpoint and the construction site.

2. The display control device according to claim 1 ,

wherein the rendering unit renders the disposed machine data by changing a position or a pose of the machine data with time.

3. The display control device according to claim 2 , further comprising:

a dynamic state setting unit that sets a dynamic state of the machine data,

wherein the rendering unit renders the disposed machine data by changing the position or the pose of the machine data with time according to the dynamic state set by the dynamic state setting unit.

4. The display control device according to claim 3 ,

wherein the machine data has a movable portion, and

wherein the rendering unit renders the disposed machine data by changing a pose of the movable portion of the disposed machine data.

5. The display control device according to claim 2 ,

wherein the machine data has a movable portion, and

wherein the rendering unit renders the disposed machine data by changing a pose of the movable portion of the disposed machine data.

6. The display control device according to claim 2 , further comprising:

an image acquisition unit that acquires an image captured by an imaging device of the portable terminal,

wherein the position/pose acquisition unit acquires a position and a pose of the first viewpoint which is a viewpoint of the imaging device.

7. The display control device according to claim 6 ,

wherein the rendering unit combines the rendered disposed machine data with the image.

8. The display control device according to claim 2 ,

wherein the position/pose acquisition unit acquires a position and a pose of the display control device.

9. The display control device according to claim 2 ,

wherein the position/pose acquisition unit acquires a position and a pose of an HMD which displays the rendered disposed machine data.

10. The display control device according to claim 1 ,

wherein the rendering unit renders one of a plurality of positions or a plurality of poses of the disposed machine data.

11. The display control device according to claim 1 ,

wherein the machine data has a movable portion, and

wherein the rendering unit renders the disposed machine data by changing a pose of the movable portion of the disposed machine data.

12. The display control device according to claim 1 , further comprising:

an image acquisition unit that acquires an image captured by an imaging device of the portable terminal,

wherein the position/pose acquisition unit acquires a position and a pose of the first viewpoint which is a viewpoint of the imaging device.

13. The display control device according to claim 12 ,

wherein the rendering unit combines the rendered disposed machine data with the image.

14. The display control device according to claim 13 , further comprising:

a site data acquisition unit that acquires the site data on the basis of the image.

15. The display control device according to claim 12 , further comprising:

a site data acquisition unit that acquires the site data on the basis of the image.

16. The display control device according to claim 1 ,

wherein the position/pose acquisition unit acquires a position and a pose of the display control device.

17. The display control device according to claim 1 ,

wherein the position/pose acquisition unit acquires a position and a pose of an HMD which displays the rendered disposed machine data.

18. The display control device according to claim 1 further comprising:

a distance calculation unit configured to measure a distance in real space between two points in the virtual space by performing steps of:

receiving a designation of coordinate on the display image as a start point of distance measurement;

disposing a straight line corresponding to the designated start point on the virtual space;

specifying a point where a straight line intersects the site data;

receiving a designation of a coordinate on the display image as an end point of distance measurement;

disposing a straight line corresponding to the designated end point on the virtual space;

specifying a point where the straight line intersects the site data;

calculating a distance connecting the specified start point and end point to each other, by multiplying a length in the virtual space between the two points specified by a predetermined gain, as a calculated length in the real space, and

rendering a straight line connecting the two points representing the measurement in real space.

19. A display control method comprising the steps of:

acquiring a position and a pose of a first viewpoint in a real space according to a position and a pose of a portable terminal comprising a display device;

disposing site data which is three-dimensional data representing a shape of a construction site in a virtual space on the basis of the position and the pose of the first viewpoint;

disposing machine data which is three-dimensional data representing a shape of a work machine at a position corresponding to the site data in the virtual space;

rendering the disposed machine data according to a second viewpoint in the virtual space; and

outputting the rendered disposed machine data to the display device,

wherein the site data disposition unit disposes the site data such that a relative position and direction between the second viewpoint and the disposed site data correspond to a relative position and direction between the first viewpoint and the construction site.

20. A program causing a computer to execute the steps of:

acquiring a position and a pose of a first viewpoint in a real space according to a position and a pose of a portable terminal comprising a display device;

disposing site data which is three-dimensional data representing a shape of a construction site in a virtual space on the basis of the position and the pose of the first viewpoint;

disposing machine data which is three-dimensional data representing a shape of a work machine at a position corresponding to the site data in the virtual space;

rendering the disposed machine data according to a second viewpoint in the virtual space; and

outputting the rendered disposed machine data to the display device,

wherein the site data disposition unit disposes the site data such that a relative position and direction between the second viewpoint and the disposed site data correspond to a relative position and direction between the first viewpoint and the construction site.

21. A display system comprising:

a position/pose measurement unit configured to measure a position and a pose of a first viewpoint in a real space according to a position and a pose of a portable terminal;

a site data disposition unit configured to dispose site data which is three-dimensional data representing a shape of a construction site in a virtual space on the basis of the position and the pose of the first viewpoint;

a machine data disposition unit configured to dispose machine data which is three-dimensional data representing a shape of a work machine at a position corresponding to the site data in the virtual space;

a rendering unit configured to render the disposed machine data according to a second viewpoint in the virtual space; and

a display unit configured to display the rendered disposed machine data wherein the display unit is provided on the portable terminal,

wherein the site data disposition unit disposes the site data such that a relative position and direction between the second viewpoint and the disposed site data correspond to a relative position and direction between the first viewpoint and the construction site.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 15, 2019
From: ONISHI, YOSHIYUKI; KURIHARA, MASAHIRO
To: KOMATSU LTD.
Reel/Frame 050715/0529 →
Priority Claims (1)
JP JP2017-138418 · Jul 14, 2017 · national
Continuity (1)
Related Publication 20200126464A1 · Apr 23, 2020