SYSTEM AND METHOD FOR THREE-DIMENSIONAL AUGMENTED REALITY GUIDANCE FOR USE OF MEDICAL EQUIPMENT
A medical guidance system providing real-time, three-dimensional ( 3 D) augmented reality (AR) feedback guidance to a novice user of medical equipment having limited medical training, to achieve improved diagnostic or treatment outcomes.
1 .- 18 . (canceled)
19 . A medical guidance system, comprising:
a medical equipment interface to a medical equipment system, wherein said medical equipment interface is capable of receiving data during a medical procedure performed by a user;
a three-dimensional guidance system (3DGS) that is capable of sensing real-time user positioning data relating to one or more of the movement, position, and orientation of at least a portion of the medical equipment system during said medical procedure performed by the user; and
an augmented reality user interface (ARUI) capable of presenting first data pertaining to real objects and second data pertaining to virtual objects to the user during at least a portion of the performance of the medical procedure, wherein the real objects include the portion of the medical equipment system and the virtual objects include the data received by the medical equipment interface.
20 . The medical guidance system of claim 19 , further comprising a computer system containing said medical equipment interface, wherein the medical equipment system comprises an ultrasound system or a non-ultrasound imaging system coupled to the computer system via the medical equipment interface.
21 . The medical guidance system of claim 20 , wherein the 3DGS senses the user's real-time movement of said portion of said medical equipment system relative to a body of a patient, and the second data include data generated prior to the performance of the medical procedure.
22 . The medical guidance system of claim 19 , further comprising:
a computer system containing said medical equipment interface to the medical equipment system, wherein said computer system is coupled to one or both of the ARUI and the 3DGS.
23 . The medical guidance system of claim 19 , wherein the 3DGS comprises one or more of a magnetic GPS system, a digital camera tracking system, an infrared camera system, an accelerometer, and a gyroscope.
24 . The medical guidance system of claim 19 , wherein the 3DGS senses real-time user positioning data by sensing at least one of:
a magnetic field generated by said at least a portion of the medical equipment system;
the movement of one or more passive visual markers coupled to one or more of a patient, a hand of the user, or a portion of the medical equipment system; and
the movement of one or more active visual markers coupled to one or more of a patient, a hand of the user, or a portion of the medical equipment system.
25 . The medical guidance system of claim 19 , wherein said first data is selected from a virtual still image, a virtual video image, sounds, or tactile information.
26 . The medical guidance system of claim 19 , wherein said second data comprise one or more of:
a virtual prompt indicating a movement correction to be performed by a user;
a virtual image or video instructing the user to change the orientation of the at least a portion of the medical equipment system to match a desired orientation;
a virtual image or video of a correct motion path to be taken by the user in performing a medical procedure;
a color-coded image or video indicating correct and incorrect portions of the user's motion in performing a medical procedure; and
an auditory instruction, virtual image, or virtual video indicating a direction for the user to move the at least a portion of the medical equipment system.
27 . The medical guidance system of claim 19 , further comprising:
a computer system containing said interface to the medical equipment system, wherein said ARUI comprises a software interface located in said computer; and
a head mounted display (HMD) coupled to said ARUI.
28 . The medical guidance system of claim 19 , wherein the three-dimensional guidance system (3DGS) is capable of sensing real-time user positioning data relating to at least a portion of the medical equipment system within a volume of the user's environment during said medical procedure performed by the user.
29 . The medical guidance system of claim 19 , wherein at least a portion of the second data are generated by a neural network.
30 . A medical guidance system, comprising:
a computer comprising a medical equipment interface to a medical equipment system, wherein said medical equipment interface is capable of receiving data during a medical procedure performed by a user;
a three-dimensional guidance system (3DGS) that is capable of sensing real-time user positioning data relating to one or more of the movement, position, and orientation of at least a portion of the medical equipment system during said medical procedure performed by the user; and
an augmented reality user interface (ARUI) capable of presenting, through an AR head mounted display (HMD), first data pertaining to real objects and second data pertaining to virtual objects to the user during at least a portion of the performance of the medical procedure, wherein the real objects include the portion of the medical equipment system and the virtual objects include the data received by the medical equipment interface.
31 . The medical guidance system of claim 30 , further comprising the AR
32 . The medical guidance system of claim 30 , wherein at least a portion of the second data are generated by a neural network.
33 . The medical guidance system of claim 32 , wherein the neural network is a convolutional neural network.
34 . A method, comprising:
receiving data during a medical procedure performed by a user using a medical equipment system;
sensing real-time user positioning data relating to one or more of the movement, position, and orientation of at least a portion of the medical equipment system during said medical procedure performed by the user; and
presenting first data pertaining to real objects and second data pertaining to virtual objects to the user during at least a portion of the performance of the medical procedure, wherein the real objects include the portion of the medical equipment system and the virtual objects include the data received by the medical equipment interface.
35 . The method of claim 34 , further comprising:
generating at least a portion of the second data prior to the performance of the medical procedure.
36 . The method of claim 34 , wherein the sensing comprises one or more of:
sensing a magnetic field generated by said at least a portion of the medical equipment system;
sensing the movement of one or more passive visual markers coupled to one or more of a patient, a hand of the user, or a portion of the medical equipment system; and
sensing the movement of one or more active visual markers coupled to one or more of a patient, a hand of the user, or a portion of the medical equipment system.
37 . The method of claim 34 , wherein the second data comprise one or more of:
a virtual prompt indicating a movement correction to be performed by a user;
a virtual image or video instructing the user to change the orientation of the at least a portion of the medical equipment system to match a desired orientation;
a virtual image or video of a correct motion path to be taken by the user in performing a medical procedure;
a color-coded image or video indicating correct and incorrect portions of the user's motion in performing a medical procedure; and
an auditory instruction, virtual image, or virtual video indicating a direction for the user to move the at least a portion of the medical equipment system.
38 . The method of claim 34 , further comprising:
generating at least a portion of the second data by a neural network.