Tabletop system for intuitive guidance in augmented reality remote video communication environment
A tabletop system for intuitive guidance in an augmented reality remote video communication environment comprises augmented reality glasses worn by a field operator, equipped with a video camera that obtains on-site, actual image information, and displaying an augmented guide on a transparent display; a tabletop receiving the actual image information from the augmented reality glasses and displaying the actual image information on a touchscreen, detecting hand motion information and instructions indicated by a remote expert in an upper space of the touchscreen and transmitting the hand motion information and instructions to the augmented reality glasses as the augmented guide.
1. A tabletop system, comprising:
augmented reality glasses worn by a field operator, equipped with a video camera that obtains on-site, actual image information, and displaying an augmented guide on a transparent display;
a tabletop displaying the actual image information, detecting hand motion information and instructions indicated by a remote expert in an upper space of the touchscreen and transmitting the hand motion information and instructions to a server; and
the server relaying data between the augmented reality glasses and the tabletop, matching coordinates of the augmented guide corresponding to the hand motion information and instructions so that the augmented guide is displayed on the corresponding position of a physical object in the actual image information, and transmitting the matched coordinates to the augmented reality glasses in real-time,
wherein the tabletop automatically switches to a drawing guide mode when a pen or an eraser is recognized, transmits moving coordinates of the pen or eraser moving on the touchscreen, and when any one of the hand of the remote expert and working tools is recognized, automatically switches to an action guide mode to transmit motion video of the hand of the remote expert and motion video of the working tool, and the tabletop recognizes each working tool, pen, eraser, and the hand of the remote expert by using 3D depth information.
2. The tabletop system of claim 1 , wherein the tabletop comprises the touchscreen displaying the actual image information, which is divided into a menu area and a working area; and
a 3D camera disposed in one side of the touchscreen and detecting hand motion information instructed by the remote expert in an upper space of the touchscreen.
3. A tabletop system, comprising:
augmented reality glasses worn by a field operator, equipped with a video camera that obtains on-site, actual image information, and displaying an augmented guide on a transparent display;
a tabletop displaying the actual image information, detecting hand motion information and instructions indicated by a remote expert in an upper space of the touchscreen and transmitting the hand motion information and instructions to a server; and
the server relaying data between the augmented reality glasses and the tabletop, matching coordinates of the augmented guide corresponding to the hand motion information and instructions so that the augmented guide is displayed on the corresponding position of a physical object in the actual image information, and transmitting the matched coordinates to the augmented reality glasses in real-time,
wherein the tabletop automatically switches to a drawing guide mode when a pen or an eraser is recognized, transmits moving coordinates of the pen or eraser moving on the touchscreen, and when any one of the hand of the remote expert and working tools is recognized, automatically switches to an action guide mode to transmit motion video of the hand of the remote expert and motion video of the working tool, and
the tabletop recognizes the hand of the remote expert by using color information and recognizes each working tool, pen, and eraser by using 3D depth information.
4. The tabletop system of claim 3 , wherein the tabletop comprises the touchscreen displaying the actual image information, which is divided into a menu area and a working area;
and
a 3D camera disposed in one side of the touchscreen and detecting hand motion information instructed by the remote expert in an upper space of the touchscreen.