Method for interaction with a user of an immersive system and device for implementing such a method
A method for interaction between at least one operator user of an immersive system, including an operator interface, and at least one expert user of an assistance system including an expert interface and a pointing tool for interacting with the operator interface. The expert interface displaying a visual depiction of a field of view of the immersed operator user and including a main window having regions for interpret a pointing action in two different ways. The method includes the expert user: orienting the operator user's gaze by introducing an orientation marker into the operator user's field of view by performing a pointing action with the pointing tool on the expert interface, and providing an indication of movement to the operator by introducing a movement marker into the operator user's field of view with a pointing action on the expert interface. An immersive collaborative device is also provided.
1 . A method for interaction between at least one operator user of an immersive system, said at least one operator user being immersed in an immersion environment a depiction of which said at least one operator user views on an operator interface, and at least one expert user of an assistance system including an expert interface and at least one pointing tool configured to interact with the operator interface, said expert interface displaying a visual depiction of a field of view of the at least one operator user of the immersive system and including a main window divided into at least two distinct continuous contiguous regions including a region in a rectangular shape, a peripheral region in the form of a frame and configured so that a pointing action is interpreted in at least two different manners, said method comprising:
an orientation step during which the at least one expert user orients a gaze of the at least one operator user by making at least one orientation marker appear in the field of view of the at least one operator user by carrying out at least one pointing action, over the peripheral region of the expert interface, by means of the at least one pointing tool on the expert interface, in order to attract the attention of the at least one operator user to a region of interest of the immersion environment, a movement step during which the at least one expert user provides an indication of movement to the at least one operator user by making at least one movement marker appear in the field of view of the at least one operator user by carrying out at least one pointing action on the expert interface, wherein during the movement step, the expert interface displays, in the main window, a planar guide corresponding to a simulated ground plane determined based on a position and an orientation of the at least one operator user, and wherein the at least one pointing action is carried out on the planar guide such that the movement marker is attached to the planar guide when the pointing action ceases, wherein said at least one pointing action of the movement step is carried out within said region in the rectangular shape.
2 . The method according to claim 1 , wherein the at least one orientation marker remains visible in the field of view of the at least one operator user as long as the at least one expert user carries out the at least one pointing action.
3 . The method according to claim 1 , wherein the at least one orientation marker disappears automatically within a predefined time period after the at least one expert user has stopped the at least one pointing action.
4 . The method according to claim 1 , wherein the at least one orientation marker is updated in real time according to the at least one pointing action carried out by the at least one expert user.
5 . The method according to claim 1 , wherein during the movement step, a planar guide is determined by the assistance system, based on a coordinate system provided by the immersive system and on an assumption that a floor of the immersion environment is at a guide height in said coordinate system, said planar guide being displayed on the expert interface to enable the at least one expert user to carry out the at least one pointing action.
6 . The method according to claim 5 , wherein the guide height is dynamically adjustable by the at least one expert user.
7 . The method according to claim 6 , wherein the planar guide is a grid or a transparent plane or a textured plane.
8 . The method according to claim 6 , wherein the planar guide is displayed on the operator interface.
9 . The method according to claim 1 , wherein during the movement step the at least one expert user proceeds with a plurality of pointing actions in order to make a trajectory appear in the field of view of the at least one operator user.
10 . The method according to claim 1 , wherein during the movement step a depiction of the field of view of the at least one operator user on the expert interface is frozen.
11 . The method according to claim 1 , wherein a movement marker created during the movement step disappears after a time period of one to five seconds after the at least one expert user has stopped a pointing action creating said movement marker.
12 . The method according to claim 1 , further comprising a highlighting step during which the at least one expert user attracts the attention of the at least one operator user to a particular element of the field of view of the at least one operator user by means of at least one highlight marker created by carrying out a one-time pointing action.
13 . The method according to claim 12 , wherein the at least one highlight marker is displayed in the field of view of the at least one operator user for a period of less than one second.
14 . The method according to claim 1 , further comprising a first annotation step, during which the at least one expert user makes at least one annotation marker appear in the field of view of the at least one operator user.
15 . The method according to claim 14 , wherein during the first annotation step, a depiction of the field of view of the at least one operator user is frozen on the expert interface.
16 . The method according to claim 14 , wherein the at least one annotation marker is displayed in the field of view of the at least one operator user for a period of less than one second.
17 . The method according to claim 14 , further comprising a second annotation step during which the at least one expert user makes annotation markers appear in the field of view of the at least one operator user.
18 . The method according to claim 17 , wherein during the second annotation step a depiction of the field of view of the at least one operator user on the expert interface is frozen.
19 . The method according to claim 18 , wherein the depiction of the field of view of the at least one operator user is frozen as long as the at least one expert user does not leave an annotation mode or as long as a predefined freezing time period has not elapsed or as long as the at least one operator user does not leave a mode.
20 . The method according to claim 17 , wherein a new window is displayed on the expert interface to display a real-time view of the at least one operator user.
21 . A collaborative immersion device comprising an immersive system, an assistance system, data transmission means between said immersive system and said assistance system, the immersive system including an operator software application implementing an operator interface and a digital processing system, the assistance system including an expert software application implementing an expert interface and a digital processing system, said expert interface displaying a visual depiction of a field of view of an operator user immersed in said immersive system, the collaborative immersion device further including at least one pointing tool and in that the expert interface includes a main window divided into at least two distinct continuous contiguous regions including a region in a rectangular shape, a peripheral region in the form of a frame and configured so that a pointing action is interpreted in at least two different manners, depending on whether the pointing action is carried out in either one of the at least two distinct contiguous regions, to make at least one orientation marker and at least one movement marker appear on the operator interface, wherein the expert interface displays, in the main window, a planar guide corresponding to a simulated ground plane determined based on a position and an orientation of the at least one operator user, and wherein the pointing action is carried out on the planar guide such that the at least one movement marker is attached to the planar guide when the pointing action ceases, wherein said pointing action for making the at least one movement marker is carried out within said region in the rectangular shape.
22 . The device according to claim 21 , wherein the digital processing system of the immersive system and the digital processing system of the assistance system form a single and unique digital processing system.
23 . The device according to claim 21 , wherein at least one of the at least two distinct contiguous regions is blurred on the expert interface.
24 . The device according to claim 21 , wherein the main window includes two distinct regions, a rectangular-shaped central region, placed in a center of said main window, and a peripheral region surrounding said rectangular-shaped central region.
25 . The device according to claim 24 , wherein a pointing action in the rectangular-shaped central region makes a marker appear on the operator interface, and in that a pointing action in the peripheral region surrounding said rectangular-shaped central region makes a marker or an annotation appear.
26 . The device according to claim 21 , wherein the expert interface includes at least one interface element allowing modifying an interpretation of a pointing action in at least one of the at least two distinct contiguous regions.
27 . The device according to claim 21 , wherein the at least one pointing tool includes a mouse or a touch surface.