Augmented-reality system for situation-related support of the interaction between a user and an engineering apparatus
The invention relates to an augmented-reality system having a mobile apparatus for the context-dependent insertion of assembly instructions. The context-dependent insertion of assembly instructions with process-optimized allocation of the necessary work steps provides situation-related support by for work sequences.
1. A situation-related support system for the interaction between a user and an apparatus comprising:
mobile equipment comprising:
a video camera for providing images of a current field of view of the user, including at least parts of the apparatus and a display, and
voice recording means for receiving voice data from the user,
a device for wireless communication coupled with said mobile equipment,
an augmented reality system comprising:
means for an analysis of video images which have been recorded by the video camera and have been wirelessly transmitted by the device for wireless communication to the augmented reality system,
means for automatically identifying one or more objects based on the analysis of the recorded video images;
means for automatically determining a particular situational context based on voice data received from the user via the voice recording means;
means for (a) automatically accessing documentation data related to the automatically identified one or more objects, wherein said documentation data related to the automatically identified one or more objects includes different data associated with different situational contexts, and (b) identifying from said documentation data the data associated with the particular situational context determined based on the voice data from the user, and
means for transmitting the context-relevant data to the device for wireless communication, wherein the display in the mobile equipment displays the context-relevant data.
2. The system according to claim 1 , wherein the apparatus is an industrial installation.
3. The system according to claim 2 , wherein the industrial installation comprises a further device for wireless communication and a communication means coupled with said augmented reality system.
4. The system according to claim 1 , wherein the voice recording means comprises a microphone.
5. The system according to claim 1 , wherein the display is located on lenses of data goggles, the video camera is an image-sensing apparatus located on the data goggles, and a microphone is located on the data goggles.
6. A situation-related support method for interaction between a user and an industrial installation, comprising the steps of:
recording video images of at least a part of said industrial installation by a mobile equipment comprising a video camera, voice recording means and a display, the at least a part of said industrial installation being within a field of view of the user;
transmitting voice data of the user from the mobile equipment to an automation system associated with said industrial installation;
wirelessly transmitting the video images from the mobile equipment to the automation system by a device,
transmitting said video images and voice data from the automation system associated with the industrial installation to an augmented reality system;
analyzing the video images in the augmented reality system,
automatically identifying one or more objects based on the analysis of the recorded video images,
automatically determining a particular situational context based on voice data received from the user via the voice recording means;
automatically accessing documentation data in the augmented reality system related to the automatically identified one or more objects, wherein said documentation data related to the automatically identified one or more objects includes different data associated with different situational contexts,
identifying from said documentation data the data associated with the determined particular situational context,
transmitting the context-relevant data, to the device for wireless communication,
displaying the context-relevant data.
7. The method according to claim 6 , wherein the voice recording means comprises a microphone.
8. The method according to claim 6 , wherein the display is arranged in data goggles and provides additional information in a field of” view of a user and wherein the context-relevant data are overlayed in the field of view of the user.
9. The method according to claim 6 , wherein the step of accessing documentation data comprises the step of accessing a remote data base or a remote expert.
10. A method for providing situational support to a user, comprising:
determining an industrial installation which requires service or repair;
recording voice data of the user,
recording video images of a portion of the industrial installation including multiple particular components;
wirelessly transmitting the video images and voice data to an augmented reality system;
analyzing the video images, identifying each of the multiple particular components from the video images, and accessing documentation data for each of said multiple particular components depending on the analysis;
determining from said documentation data and said voice data from said user a context-dependent instruction sequence, the context-dependent instruction sequence comprising instructions regarding each of the multiple particular components, and being pre-associated with a particular situational context determined based on the voice data from said user; and
displaying the context-dependent instruction sequence such that the user may view the context-dependent instructions substantially simultaneously with the component of the industrial installation, and such that the instructions regarding each of the multiple particular components are displayed in association with each of the multiple particular components simultaneously.
11. The method of claim 10 , further comprising an intelligent tool operated by the user, wherein the method further comprises the step of receiving information from said intelligent tool and instructing the user how to operate said intelligent tool.
12. The method of claim 10 , wherein the step of analyzing the video images comprises the step of determining at least one point of said industrial installation within an video image and displaying instructions in a field of view of the user relating to said one point.
13. The method of claim 12 , wherein said instructions are displayed in a window and a graphical connection is drawn from said window to said one point.