Granted Patent B2

US 12,231,502 · App. 17/835,619 · Granted Feb 18, 2025

Inventor: Tae Jin Ha (Gimpo-si, KR)

Assignee: VIRNECT INC.

H04L67/125B25J13/00G06F3/011G06F3/0481G06T19/006G06T2219/024

Abstract

Claims (46)

1. A method for a remote collaboration, as a method for providing an augmented reality (AR)-based remote collaboration between a robot located in a worksite, a field worker terminal, and a remote administrator terminal located outside the worksite, the method comprising:

acquiring a captured image including a field image captured by a robot located at the worksite or a captured image including a user image captured by the field worker terminal;

displaying the captured image of the worksite;

generating virtual content based on an input of a remote administrator and a field worker with respect to the displayed captured image; and

displaying an AR image in which the virtual content is augmented on the displayed captured image,

wherein the displaying of the AR image in which the virtual content is augmented on the displayed captured image includes at least one of displaying the AR image in the field worker terminal; displaying the AR image in the remote administrator terminal; and the AR image on a display device disposed on the robot,

wherein the display device is a computing device detachably disposed in a body of the robot and connected to the robot for wireless or wired communication, and the acquiring of the captured image includes acquiring a sub-field image captured by the display device detachable from the robot, and

wherein the generating of the virtual content based on an input of the remote administrator and the field worker for the displayed captured image includes:

displaying the field image in the remote administrator terminal and generating the virtual content according to the input of the remote administrator from the displayed field image, and

the displaying of the AR image on the display device disposed on the robot includes displaying an AR image in which the virtual content is augmented on the field image captured by the robot.

2. The method of claim 1 , wherein

the generating of the virtual content based on the input of the remote administrator and the field worker with respect to the displayed captured image includes:

acquiring virtual content that visualizes work guidance to be transferred from the remote administrator terminal to the field worker,

wherein the work guidance includes at least one of text, pointing, image, drawing, and voice data based on a user input of the remote administrator terminal.

3. The method of claim 2 , wherein

the generating of the virtual content based on the input of the remote administrator and the field worker with respect to the displayed captured image includes:

obtaining virtual content that visualizes a work feedback to be displayed on the remote administrator terminal from the field worker terminal,

wherein the work feedback includes at least one of text, pointing, image, drawing, and voice data based on a user input of the field worker terminal.

4. The method of claim 1 , further comprising:

separating the display device from the robot;

activating a real-time movement direction input interface for controlling movement of the robot in the display device;

controlling the robot to move based on a user input to the real-time movement direction input interface; and

acquiring a real-time field image captured by the moved robot and displaying the real-time field image on the display device and the remote administrator terminal.

5. The method of claim 1 , further comprising:

receiving work guidance for a point or object other than the real-time field image from the remote administrator terminal; and

augmenting and displaying virtual content for guiding a location of the work guidance in the real-time field image.

6. The method of claim 5 , wherein

the virtual content for guiding a position of the work guidance is a navigation image indicating a direction in which the robot is to move in order to image a position of the point or object.

7. The method of claim 1 , further comprising:

receiving a work guidance for a past field image captured by the robot before moving from the remote administrator terminal; and

comparing the real-time field image captured by the robot with the past field image and determining a display position of the work guidance in the real-time field image.

8. The method of claim 1 , further comprising:

displaying a base map for the worksite in the remote administrator terminal;

receiving a user input from the remote administrator terminal for the displayed base map; and

augmenting and displaying virtual content on the captured image based on the received user input.

9. The method of claim 8 , wherein

the receiving of a user input from the remote administrator terminal for the displayed base map includes:

receiving a work guidance for a predetermined point or a predetermined object in the base map, and

the augmenting and displaying of virtual content on the captured image based on the received user input includes:

acquiring the captured image of a target area including the predetermined point or the predetermined object;

determining a display position of the work guidance based on the acquired captured image; and

outputting the captured image augmented with the virtual content representing the work guidance based on the determined display position.

10. The method of claim 9 , wherein

the acquiring of the captured image for the area including the predetermined point or the predetermined object includes:

controlling an image capture direction of the display device disposed on the robot to image the target area; and

controlling movement of the robot to image the target area.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (38)