System and method using augmented reality with shape alignment for medical device placement
Methods and techniques for using an augmented reality device for determining an orientation of an instrument for inserting a medical device are provided. One such method includes simulating an insertion point and an orientation of a simulated surgical hardware installation on a diagnostic representation of a bone, and using an augmented reality based electronic device to align an instrument for inserting a surgical hardware installation at a desired orientation through an insertion point of the bone by displaying visual indicia indicating the insertion point and the orientation of the simulated surgical hardware installation. Disclosed herein is also a system and method having a concentric shape alignment system for medical device placement in the body, such as in a bone.
1 . A method for orienting an electronic device to align an instrument for installation of a surgical hardware in a body part of a patient, the method comprising:
simulating, by a computing device, an insertion point of the instrument and a three-dimensional orientation of a simulated surgical hardware installation on a diagnostic representation of the body part, wherein simulating the three-dimensional orientation of the simulated surgical hardware installation includes:
receiving, by the computing device, a first user input in a first view of the diagnostic representation of the body part; and
receiving, by the computing device, a second user input in a second view of the diagnostic representation of the body part, the second view orthogonal to the first view; and
displaying, by the computing device, a graphical representation of the simulated three-dimensional orientation for inserting the surgical hardware in the body part, and a graphical representation of a current orientation of the electronic device on an augmented reality display or virtual reality display superimposed over the body part, the graphical representation of a current orientation of the electronic device identified by at least one orientation sensor of the electronic device, wherein the graphical representation of the simulated three-dimensional orientation and the graphical representation of a current orientation of the electronic device is used to align the instrument for insertion of the surgical hardware.
2 . The method of claim 1 , wherein the body part of the patient is a bone.
3 . The method of claim 1 , further comprising indicating, by the computing device, when the electronic device is oriented to align the instrument for inserting the surgical hardware in the body part within a threshold of the simulated three-dimensional orientation.
4 . The method of claim 3 , wherein indicating when an orientation of the electronic device is within the threshold of the simulated three-dimensional orientation comprises displaying additional visual indicia indicating when the orientation of the electronic device is within the threshold of the simulated three-dimensional orientation.
5 . The method of claim 3 , wherein indicating when an orientation of the electronic device is within the threshold of the simulated three-dimensional orientation comprises audibly indicating when the orientation of the electronic device is within the threshold of the simulated three-dimensional orientation.
6 . The method of claim 1 , wherein the graphical representation of the simulated three-dimensional orientation for inserting the surgical hardware in the body part includes a graphical representation of an orientation of the instrument for inserting the surgical hardware superimposed on the body part.
7 . The method of claim 1 , wherein the graphical representation of the simulated three-dimensional orientation for inserting the surgical hardware in the body part includes a graphical representation of an orientation of the surgical hardware superimposed on the body part.
8 . The method of claim 1 , wherein displaying the graphical representation of the simulated three-dimensional orientation for inserting the surgical hardware in the body part includes providing a three-dimensional visual representation on a display of an augmented reality device, the augmented reality device wearable by a user.
9 . The method of claim 1 , wherein simulating the insertion point and the three-dimensional orientation of the simulated surgical hardware installation on the diagnostic representation of the body part further comprises:
acquiring, by the computing device, the diagnostic representation of the body part respectively in the first view and in the second view;
aligning, by the computing device, the diagnostic representation of the body part with a reference;
designating, by the computing device, the insertion point of the simulated surgical hardware installation on the diagnostic representation of the body part based on a respective user-specified insertion location in the first view and in the second view; and
designating, by the computing device, the simulated three-dimensional orientation of the simulated surgical hardware installation on the diagnostic representation of the body part relative to the reference based on a respective user specified insertion orientation in the first view and in the second view.
10 . The method of claim 1 , wherein displaying a graphical representation of the simulated three-dimensional orientation for inserting the surgical hardware in the body part includes providing on a display of the electronic device a first set of concentric circles indicating the simulated three-dimensional orientation and a second set of concentric circles indicating a current orientation of the electronic device, wherein the orientation of the electronic device corresponds to the orientation of the instrument.
11 . The method of claim 1 wherein the diagnostic representation is an axial CT scan of the body part.
12 . A method for generating a three-dimensional angle for use in positioning a medical tool, the method comprising:
obtaining, by a computing device, a first image of a target;
obtaining, by the computing device, a second image of the target, the second image depicting the target in a plane at a known angle to the first image;
defining, by the computing device, a first insertion angle in the first image;
defining, by the computing device, a second insertion angle in the second image;
generating, by the computing device, the three-dimensional angle based at least in part on the first insertion angle of the first image and the second insertion angle of the second image, wherein the three-dimensional angle comprises at least one or more of an insertion transverse angle, an insertion sagittal angle, or an insertion coronal angle for receiving the medical tool;
determining, by the computing device, a present three-dimensional orientation of an electronic device based on data obtained by at least one orientation sensor of the electronic device; and
producing, by the computing device, a notification when the present three-dimensional orientation of the device aligns with the three-dimensional angle on an augmented reality display or virtual reality display, wherein the three-dimensional orientation is used to insert the medical tool into a patient, responsive to the notification.
13 . The method of claim 12 , wherein defining a first insertion angle in the first image includes defining an insertion point.