SYSTEM AND METHOD FOR AUGMENTED REALITY GUIDANCE FOR USE OF EQUPMENT SYSTEMS

1 . A method for providing real-time augmented reality (AR) feedback guidance to a user of an equipment system, the method comprising:

sensing real-time user positioning data relating to one or more of the three-dimensional movement, position, and orientation of at least a portion of an equipment system within a volume of a user's environment during an equipment procedure performed by the user using the equipment system;

retrieving from a library stored reference positioning data relating to one or more of the three-dimensional movement, position, and orientation of at least a portion of the equipment system during a reference equipment procedure;

comparing the sensed real-time user positioning data to the retrieved reference positioning data;

generating real-time position-based AR feedback based on the comparison of the sensed real-time user positioning data to the retrieved reference positioning data;

retrieving from the library instructions for performing one or more actions of the equipment procedure;

providing at least one of the real-time position-based AR feedback and the instructions to the user via an augmented reality user interface (ARUI);

receiving a user input selected from a voice command, an eye movement, a hand movement, or two or more thereof; and

modifying at least one instruction, the real-time position-based AR feedback, or both in response to the user input.

2 . The method of claim 1 , wherein the ARUI is configured to present the data providing the real-time position-based AR feedback and the instructions to the user through a head mounted display (HMD) worn by the user.

3 . The method of claim 2 , wherein the ARUI is configured to present the real-time position-based AR feedback to the user visually in a HMD relative to at least a portion of the equipment system.

4 . The method of claim 1 , wherein the receiving the user input consists of receiving the voice command.

5 . The method of claim 1 , wherein the receiving the user input comprises receiving the eye movement, the hand movement, or both.

6 . The method of claim 1 , further comprising:

identifying the at least one instruction, the position-based AR feedback, or both to be modified based on an input selected from a first voice command, a first eye movement, a first-hand movement, or two or more thereof;

wherein the modifying is based on a second voice command, a second eye movement, a second hand movement, or two or more thereof.

7 . The method of claim 6 , wherein the identifying the one of the one or more augmented reality elements is based solely on a first voice command; and the modifying is based solely on a second voice command.

8 . A guidance system for providing real-time augmented reality (AR) feedback guidance to a user of an equipment system, the guidance system comprising:

an augmented reality user interface (ARUI) for presenting data pertaining to both real and virtual objects to the user during at least a portion of the performance of an equipment procedure making use of the equipment system;

a position sensing system that is capable of sensing real-time user positioning data relating to one or more of the movement, position, and orientation of at least a portion of the equipment system during said procedure performed by the user;

a library containing 1) stored reference positioning data relating to one or more of the three-dimensional movement, position, and orientation of at least a portion of the equipment system during a reference procedure, and 2) instructions for performing one or more actions of the equipment procedure;

a machine learning module (MLM) for providing 1) real-time position-based AR feedback to the user based on the sensed user positioning data and the reference positioning data, and 2) instructions to the user via the ARUI; and

an input module for receiving, from the user, an input selected from a voice command, an eye movement, a hand movement, or two or more thereof;

wherein the MLM modifies at least one instruction, the real-time position-based AR feedback, or both in response to the input.

9 . The guidance system of claim 8 , wherein the ARUI is configured to present the data providing the real-time position-based AR feedback and the instructions to the user through a head mounted display (HMD) worn by the user.

10 . The guidance system of claim 9 , wherein the ARUI is configured to present the real-time position-based AR feedback to the user visually in a HMD relative to at least a portion of the equipment system.

11 . The guidance system of claim 8 , wherein the input module is configured to only receive the voice command.

12 . The guidance system of claim 8 , wherein input module is configured to receive the eye movement, the hand movement, or both.

13 . The guidance system of claim 8 , wherein the MLM is further configured to identify the at least one instruction, the position-based AR feedback, or both to be modified based on an input selected from a first voice command, a first eye movement, a first-hand movement, or two or more thereof; wherein the modifying is based on a second voice command, a second eye movement, a second hand movement, or two or more thereof; and to modify based on a second voice command, a second eye movement, a second hand movement, or two or more thereof.

14 . The guidance system of claim 13 , wherein the MLM is configured to identify the at least one instruction, the position-based AR feedback, or both to be modified based solely on a first voice command; and the modifying is based solely on a second voice command.

15 . A system, comprising:

an augmented reality display configured to present data pertaining to both real and virtual objects to the user during at least a portion of the performance of an equipment procedure making use of an equipment system;

a position sensing system configured to sense real-time user positioning data relating to one or more of the movement, position, and orientation of a tool of said equipment system during said procedure performed by the user;

a memory containing 1) stored reference positioning data relating to one or more of the three-dimensional movement, position, and orientation of at least a portion of the equipment system during a reference procedure, and 2) instructions for performing one or more actions of the equipment procedure;

a controller configured to provide 1) real-time position-based AR feedback to the user based on the sensed user positioning data and the reference positioning data, and 2) instructions to the user via the augmented reality display; and

a user input module configured to receive, from the user, an input selected from a voice command, an eye movement, a hand movement, or two or more thereof;

wherein the controller is configured to modify at least one instruction, the real-time position-based AR feedback, or both augmented reality element in response to the input.

16 . The system of claim 15 , wherein the augmented reality display is a head mounted display (HMD) worn by the user.

17 . The system of claim 15 , wherein the user input module is configured to only receive the voice command.

18 . The system of claim 15 , wherein the user input module is configured to receive the eye movement, the hand movement, or both.

19 . The system of claim 15 , wherein the controller is further configured to identify the at least one instruction, the position-based AR feedback, or both to be modified based on an input selected from a first voice command, a first eye movement, a first-hand movement, or two or more thereof; wherein the modifying is based on a second voice command, a second eye movement, a second hand movement, or two or more thereof; and to modify based on a second voice command, a second eye movement, a second hand movement, or two or more thereof.

20 . The system of claim 19 , wherein the controller is configured to identify the at least one instruction, the position-based AR feedback, or both to be modified based solely on a first voice command; and the modifying is based solely on a second voice command.