Systems, methods, apparatuses, and computer-readable media for image guided surgery
Presented herein are methods, systems, devices, and computer-readable media for image guided surgery. The systems herein allow a physician to use multiple instruments for a surgery and simultaneously provide image-guidance data for those instruments. Various embodiments disclosed herein provide information to physicians about procedures they are performing, the devices (such as ablation needles, ultrasound wands or probes, scalpels, cauterizers, etc.) they are using during the procedure, the relative emplacements or poses of these devices, prediction information for those devices, and other information. Some embodiments provide useful information about 3D data sets. Additionally, some embodiments provide for quickly calibratable surgical instruments or attachments for surgical instruments.
1. A method, comprising:
determining, with one or more processors, real-time position and orientation of a first medical device;
determining, with the one or more processors, real-time position and orientation of a second medical device;
determining position and orientation of a 2D image area based at least in part on the determined real-time position and orientation of the second medical device;
receiving in real-time image data for the 2D image area from the second medical device;
determining a perspective view of the 2D image area in a virtual 3D space based at least in part on a relative orientation of the second medical device with respect to an expected location of a user, wherein the expected location of the user is a fixed location in front of one or more displays; and
causing the one or more displays to display in real-time a perspective view of an image corresponding to the image data in the virtual 3D space based at least in part on the determined perspective view of the 2D image area.
2. The method of claim 1 , further comprising causing the one or more displays to display in real-time a perspective view of a virtual medical device corresponding to the first medical device based at least in part on the relative orientation of the first medical device with respect to the expected location of the user.
3. The method of claim 1 , further comprising causing the one or more displays to display a 2D view of the image corresponding to the image data.
4. The method of claim 1 , further comprising causing the one or more displays to display a graphic indicating an intersection between a trajectory of the first medical device and the 2D image area.
5. The method of claim 1 , further comprising causing the one or more displays to display a graphic indicating an intersection between a trajectory of the first medical device and a location on a 2D plane corresponding to the 2D image area, wherein the location on the 2D plane is outside the 2D image area.
6. The method of claim 1 , wherein the first medical device comprises an ablation needle, and wherein the method further comprises causing the one or more displays to display in real-time a perspective view of an ablation volume for the ablation needle in the virtual 3D space based at least in part on the relative orientation of the first medical device with respect to the expected location of the user.
7. The method of claim 1 , further comprising:
receiving an indication of a point of interest;
marking in an underlying 3D model an intersection of a trajectory of the first medical device and a 2D plane corresponding to the 2D image area as the point of interest; and
causing the one or more displays to display in real-time a perspective view of the point of interest in the virtual 3D space based at least in part on the determined perspective view of the 2D image area.
8. The method of claim 1 , wherein the first medical device comprises a needle and the second medical device comprises an imaging device.
9. A system, comprising:
a computing device comprising one or more processors, the computing device configured to:
determine real-time position and orientation of a first medical device based at least in part on data received from a first pose sensor;
determine real-time position and orientation of a second medical device based at least in part on data received from a second pose sensor;
determine position and orientation of a 2D image area based at least in part on the determined real-time position and orientation of the second medical device;
receive image data for the 2D image area from the second medical device;
determine a perspective view of the 2D image area in a virtual 3D space based at least in part on a relative orientation of the second medical device with respect to an expected location of a user, wherein the expected location of the user is a fixed location in front of one or more displays; and
cause the one or more displays to display in real-time a perspective view of an image corresponding to the image data in the virtual 3D space based at least in part on the determined perspective view of the 2D image area.
10. The system of claim 9 , wherein the computing device is further configured to cause the one or more displays to display in real-time a perspective view of a virtual medical device corresponding to the first medical device based at least in part on the relative orientation of the first medical device with respect to the expected location of the user.
11. The system of claim 9 , wherein the computing device is further configured to cause the one or more displays to display a 2D view of the image corresponding to the image data.
12. The system of claim 9 , wherein the computing device is further configured to cause the one or more displays to display a graphic indicating an intersection between a trajectory of the first medical device and the 2D image area.
13. The system of claim 9 , wherein the computing device is further configured to cause the one or more displays to display a graphic indicating an intersection between a trajectory of the first medical device and a location on a 2D plane corresponding to the 2D image area, wherein the location on the 2D plane is outside the 2D image area.
14. The system of claim 9 , wherein the first medical device comprises an ablation needle and the computing device is further configured to cause the one or more displays to display in real-time a perspective view of an ablation volume for the ablation needle in the virtual 3D space based at least in part on the relative orientation of the first medical device with respect to the expected location of the user.
15. The system of claim 9 , wherein the computing device is further configured to:
receive an indication of a point of interest;
mark, in an underlying 3D model, an intersection of a trajectory of the first medical device and a 2D plane corresponding to the 2D image area as the point of interest; and
cause the one or more displays to display in real-time a perspective view of the point of interest in the virtual 3D space based at least in part on the determined perspective view of the 2D image area.
16. The system of claim 9 , wherein the first medical device comprises a needle and the second medical device comprises an imaging device.
17. A non-transitory computer-readable medium comprising computer-executable instructions, said instructions, when executed, cause one or more processors to:
determine real-time position and orientation of a first medical device based at least in part on data received from a first pose sensor;
determine real-time position and orientation of a second medical device based at least in part on data received from a second pose sensor;
determine position and orientation of a 2D image area based at least in part on the determined real-time position and orientation of the second medical device;
receive image data for the 2D image area from the second medical device;
determine a perspective view of the 2D image area in a virtual 3D space based at least in part on a relative orientation of the second medical device with respect to an expected location of a user, wherein the expected location of the user is a fixed location in front of one or more displays; and
cause the one or more displays to display in real-time a perspective view of an image corresponding to the image data in the virtual 3D space based at least in part on the determined perspective view of the 2D image area.
18. The non-transitory computer-readable medium of claim 17 , wherein said instructions, when executed, further cause one or more processors to:
receive an indication of a point of interest;
mark in an underlying 3D model an intersection of a trajectory of the first medical device and a 2D plane corresponding to the 2D image area as the point of interest; and
cause the one or more displays to display in real-time a perspective view of the point of interest in the virtual 3D space based at least in part on the determined perspective view of the 2D image area.
19. The non-transitory computer-readable medium of claim 17 , wherein said instructions, when executed, further cause one or more processors to cause the one or more displays to display a 2D view of the image corresponding to the image data.
20. The non-transitory computer-readable medium of claim 17 , wherein said instructions, when executed, further cause one or more processors to cause the one or more displays to display a graphic indicating an intersection between a trajectory of the first medical device and a location on a 2D plane corresponding to the 2D image area, wherein the location on the 2D plane is outside the 2D image area.