System and method for performing physical exam on XR-based simulated patients
Systems and methods for medical training are provided. In some embodiments, systems and methods use computer generated simulated patients through augmented reality. In some embodiments, systems and methods enable medical training that allows a user to interact with the simulated patient which provides simulated visual, audio, or other user perceivable reactions. In some embodiments, hand tracking technology is used in a unique way to detect collisions with a virtual object to result in naturally appearing physical exam maneuvers to simulate the physical examination (e.g., auscultation, percussion, palpation stages of a physical examination) of the simulated patient.
1. A system comprising:
an augmented reality display device having a controller, the augmented reality device configured to:
generate a display of a virtual patient;
track the hands of a user to enable the user to perform one or more physical examinations on the virtual patient; and
responsive to the tracking the hands of the user during the performance of the one or more physical examinations on the virtual patient, generate one or more simulated audio or visual reactions by the virtual patient that can be perceived by the user while viewing the virtual patient via the generated display to simulate reactions of a real patient undergoing a similar physical examination based on one or more of collision direction, velocity, and depth of the hand of the user relative to the virtual patient.
2. The system of claim 1 , wherein the one or more physical examinations on the virtual patient includes auscultation.
3. The system of claim 2 , wherein the controller device is further configured to provide a virtual stethoscope controllable by the user.
4. The system of claim 2 , wherein the controller device is further configured to:
provide a plurality of auscultation sound points on the virtual patent; and
responsive to the user's hand being in the proximity of one or more of the auscultation sound points, generating the response by the virtual patient that can be perceived by the user.
5. The system of claim 1 , wherein the one or more physical examinations on the virtual patient includes palpation.
6. The system of claim 1 , wherein the one or more physical examinations on the virtual patient includes percussion.
7. The system of claim 1 , wherein the one or more physical examinations on the virtual patient includes allowing the user to manipulate a virtual body part through a given range of motion and observe a reaction of the virtual patient.
8. A method of simulating a medical examination comprising:
configuring an augmented reality display device to display a virtual patient;
tracking the hands of a user to enable the user to perform one or more physical examinations on the virtual patient; and
responsive to the tracking the hands of the user during the performance of the one or more physical examinations on the virtual patient, generating one or more simulated audio or visual reactions by the virtual patient that can be perceived by the user while viewing the virtual patient via the generated display to simulate reactions of a real patient undergoing a similar physical examination based on one or more of collision direction, velocity, and depth of the hand of the user relative to the virtual patient.
9. The method of claim 8 , wherein the one or more physical examinations on the virtual patient includes auscultation.
10. The method of claim 9 , further comprising providing a virtual stethoscope controllable by the user.
11. The method of claim 9 , further comprising:
providing a plurality of auscultation sound points on the virtual patent; and
responsive to the user's hand being in the proximity of one or more of the auscultation sound points, generating the response by the virtual patient that can be perceived by the user.
12. The method of claim 8 , wherein the one or more physical examinations on the virtual patient includes palpation.
13. The method of claim 8 , wherein the one or more physical examinations on the virtual patient includes percussion.
14. The method of claim 8 , wherein the one or more physical examinations on the virtual patient includes allowing the user to manipulate a virtual body part through a given range of motion and observe a reaction of the virtual patient.
15. A method of training a user to conduct physical examinations comprising:
providing the user with an augmented reality display device;
configuring the augmented reality display device to display a virtual patient;
tracking the hands of the user to enable the user to perform one or more physical examinations on the virtual patient; and
responsive to the tracking the hands of the user during the performance of the one or more physical examinations on the virtual patient, generating one or more simulated audio or visual reactions by the virtual patient that can be perceived by the user while viewing the virtual patient via the generated display to simulate reactions of a real patient undergoing a similar physical examination based on one or more of collision direction, velocity, and depth of the hand of the user relative to the virtual patient.
16. The method of claim 15 , wherein the one or more physical examinations on the virtual patient includes auscultation.
17. The method of claim 16 , further comprising providing a virtual stethoscope controllable by the user.
18. The method of claim 16 , further comprising:
providing a plurality of auscultation sound points on the virtual patent; and
responsive to the user's hand being in the proximity of one or more of the auscultation sound points, generating the response by the virtual patient that can be perceived by the user.
19. The method of claim 15 , wherein the one or more physical examinations on the virtual patient includes palpation.
20. The method of claim 15 , wherein the one or more physical examinations on the virtual patient includes percussion.