SYSTEM AND METHOD FOR AUGMENTED REALITY GUIDANCE FOR USE OF EQUIPMENT SYSTEMS
Augmented reality (AR) guidance systems providing procedure instruction data to a user using an equipment system to perform a procedure. Methods for use of the AR guidance systems.
1 . A method for providing real-time augmented reality (AR) feedback guidance to a user of an equipment system, the method comprising:
sensing real-time user positioning data relating to one or more of the three-dimensional movement, position, and orientation of at least a portion of an equipment system within a volume of a user's environment during an equipment procedure performed by the user using the equipment system;
retrieving from a library stored reference positioning data relating to one or more of the three-dimensional movement, position, and orientation of at least a portion of the equipment system during a reference equipment procedure;
comparing the sensed real-time user positioning data to the retrieved reference positioning data;
generating real-time position-based AR feedback based on the comparison of the sensed real-time user positioning data to the retrieved reference positioning data;
retrieving from the library instructions for performing one or more actions of the equipment procedure;
providing at least one of the real-time position-based AR feedback and the instructions to the user via an augmented reality user interface (ARUI; and
recording, during the equipment procedure, at least one of the user or a target on which the equipment procedure is performed.
2 . The method of claim 1 , wherein the ARUI is configured to present the data providing the real-time position-based AR feedback and the instructions to the user through a head mounted display (HMD) worn by the user.
3 . The method of claim 2 , wherein the ARUI is configured to present the real-time position-based AR feedback to the user visually in a HMD relative to at least a portion of the equipment system.
4 . The method of claim 1 , wherein the recording is performed by an optical camera or an infrared camera.
5 . The method of claim 1 , wherein the recording is performed by an ultraviolet light camera or a microphone.
6 . The method of claim 1 , further comprising:
communicating the recording of the user and/or the target to a system other than the equipment system.
7 . The method of claim 6 , wherein the system other than the equipment system is a server located on a public network, a private network, or a virtual private network (VPN).
8 . A guidance system for providing real-time augmented reality (AR) feedback guidance to a user of an equipment system, the guidance system comprising:
an augmented reality user interface (ARUI) for presenting data pertaining to both real and virtual objects to the user during at least a portion of the performance of an equipment procedure making use of the equipment system;
a position sensing system that is capable of sensing real-time user positioning data relating to one or more of the movement, position, and orientation of at least a portion of the equipment system during said procedure performed by the user;
a library containing 1) stored reference positioning data relating to one or more of the three-dimensional movement, position, and orientation of at least a portion of the equipment system during a reference procedure, and 2) instructions for performing one or more actions of the equipment procedure;
a machine learning module (MLM) for providing 1) real-time position-based AR feedback to the user based on the sensed user positioning data and the reference positioning data, and 2) instructions to the user via the ARUI; and
a recording device for recording of at least one of the user or a target on which the procedure is performed.
9 . The guidance system of claim 8 , wherein the ARUI is configured to present the data providing the real-time position-based AR feedback and the instructions to the user through a head mounted display (HMD ) worn by the user.
10 . The guidance system of claim 9 , wherein the ARUI is configured to present the real-time position-based AR feedback to the user visually in a HMD relative to at least a portion of the equipment system.
11 . The guidance system of claim 8 , wherein the recording device is an optical camera or an infrared camera.
12 . The guidance system of claim 8 , wherein the recording device is an ultraviolet light camera or a microphone.
13 . The guidance system of claim 8 , further comprising:
a communication interface for providing the recording of the user and/or the target to a system other than the equipment system.
14 . The guidance system of claim 13 , wherein the system other than the equipment system is a server located on a public network, a private network, or a virtual private network (VPN).
15 . A system, comprising:
an augmented reality display configured to present data pertaining to both real and virtual objects to the user during at least a portion of the performance of an equipment procedure making use of an equipment system;
a position sensing system configured to sense real-time user positioning data relating to one or more of the movement, position, and orientation of a tool of said equipment system during said procedure performed by the user;
a memory containing 1) stored reference positioning data relating to one or more of the three-dimensional movement, position, and orientation of at least a portion of the equipment system during a reference procedure, and 2) instructions for performing one or more actions of the equipment procedure;
a controller configured to provide 1) real-time position-based AR feedback to the user based on the sensed user positioning data and the reference positioning data, and 2) instructions to the user via the augmented reality display; and
a recording device configured to record at least one of the user or a target on which the procedure is performed.
16 . The system of claim 15 , wherein the augmented reality display is a head mounted display (HMD) worn by the user.
17 . The system of claim 15 , wherein the recording device is an optical camera or an infrared camera.
18 . The system of claim 15 , wherein the recording device is an ultraviolet light camera or a microphone.
19 . The system of claim 15 , further comprising:
a communication interface for providing the recording of the user and/or the target to a system other than the equipment system.
20 . The system of claim 19 , wherein the system other than the equipment system is a server located on a public network, a private network, or a virtual private network (VPN).