Tracking methods for image-guided surgery
Apparatus and methods are described including tracking a tool portion and a patient marker from a first line of sight, using a first tracking device disposed upon a first head-mounted device that includes a display. The tool portion and the patient marker are tracked from a second line of sight, using a second tracking device. When a portion of the patient marker and the tool portion are both within the first line of sight, an augmented reality image is generated upon the first display based upon data received from the first tracking device and without using data from the second tracking device. When at least the patient marker portion and the tool portion are not both within the first line of sight, a virtual image of the tool and anatomy of the patient is generated using data received from the second tracking device. Other applications are also described.
1. A method of generating images for augmented reality surgery using multiple tracking devices, the method comprising:
determining a first position of a tool with respect to an anatomy of a patient using data from tracking of both a patient marker and the tool by a first tracking device of a first head-mounted device,
wherein the first head-mounted device comprises a first head-mounted display, and
wherein the first tracking device comprises a first camera;
generating, by at least one computer processor using the determined first position of the tool with respect to the anatomy of the patient, a first augmented reality image for display by the first head-mounted display,
wherein the first augmented reality image comprises a virtual image of the tool aligned with a virtual image of the anatomy of the patient;
detecting at least one of (i) a line of sight between the first tracking device and the patient marker is blocked and a line of sight between the first tracking device and the tool is not blocked, or (ii) a line of sight between the first tracking device and the tool is blocked and a line of sight between the first tracking device and the patient marker is not blocked;
in response to the detecting at least one of (i) the line of sight between the first tracking device and the patient marker is blocked and the line of sight between the first tracking device and the tool is not blocked, or (ii) the line of sight between the first tracking device and the tool is blocked and the line of sight between the first tracking device and the patient marker is not blocked, determining a second position of the tool with respect to the anatomy of the patient using data from a second tracking device that comprises a second camera; and
generating, by the at least one computer processor using the determined second position of the tool with respect to the anatomy of the patient, a second augmented reality image for display by the first head-mounted display,
wherein the second augmented reality image comprises a virtual image of the tool aligned with a virtual image of the anatomy of the patient.
2. The method of claim 1 ,
wherein the second tracking device is part of a second head-mounted device, and
wherein the second head-mounted device comprises a second head-mounted display.
3. The method of claim 2 , further comprising generating a further augmented reality image for display by the second head-mounted display.
4. The method of claim 1 , wherein the second tracking device is disposed in a stationary position.
5. The method of claim 1 , wherein the tracking of the tool comprises tracking a tool marker.
6. The method of claim 1 , wherein the first tracking device further comprises a light source.
7. The method of claim 1 , wherein the at least one computer processor is integrated within the first head-mounted device.
8. The method of claim 1 , wherein the at least one computer processor is disposed externally to the first head-mounted device.
9. The method of claim 1 , further comprising displaying the first augmented reality image by the first head-mounted display, with the virtual image of the anatomy of the patient of the first augmented reality image being aligned with actual patient anatomy based upon data from tracking of at least the patient marker by the first tracking device.
10. The method of claim 1 , further comprising, responsive to detecting that the line of sight between the first tracking device and the patient marker is blocked, displaying the second augmented reality image by the first head-mounted display, with the virtual image of the anatomy of the patient of the second augmented reality image being positioned based upon data from tracking of the patient marker by the first tracking device at a time when the line of sight between the first tracking device and the patient marker was not blocked.
11. The method of claim 1 , further comprising, responsive to detecting that the line of sight between the first tracking device and the patient marker is blocked, displaying the second augmented reality image by the first head-mounted display, with the virtual image of the anatomy of the patient of the second augmented reality image being aligned with actual patient anatomy based upon data from tracking of movements of the first head-mounted device using an inertial measurement unit of the first head-mounted device.
12. The method of claim 1 , wherein the detecting at least one of (i) the line of sight between the first tracking device and the patient marker is blocked and the line of sight between the first tracking device and the tool is not blocked, or (ii) the line of sight between the first tracking device and the tool is blocked and the line of sight between the first tracking device and the patient marker is not blocked comprises detecting that the line of sight between the first tracking device and the patient marker is blocked and the line of sight between the first tracking device and the tool is not blocked.
13. The method of claim 1 , wherein the detecting at least one of (i) the line of sight between the first tracking device and the patient marker is blocked and the line of sight between the first tracking device and the tool is not blocked, or (ii) the line of sight between the first tracking device and the tool is blocked and the line of sight between the first tracking device and the patient marker is not blocked comprises detecting that the line of sight between the first tracking device and the tool is blocked and the line of sight between the first tracking device and the patient marker is not blocked.
14. A method of generating images for augmented reality surgery using multiple tracking devices, the method comprising:
determining a first position of a first head-mounted device with respect to anatomy of a patient, and a first position of a tool with respect to the anatomy of the patient, using data from tracking of a patient marker and the tool by a first tracking device of the first head-mounted device,
wherein the first head-mounted device comprises a first head-mounted display;
generating, by at least one computer processor, for display by the first head-mounted display, a first augmented reality image, the first augmented reality image comprising a virtual image of the tool aligned with a virtual image of the anatomy of the patient, and the first augmented reality image being aligned with actual patient anatomy based upon the determined first position of the first head-mounted device;
detecting that a line of sight between the first tracking device and the patient marker is blocked and a line of sight between the first tracking device and the tool is not blocked;
in response to the detecting that the line of sight between the first tracking device and the patient marker is blocked and the line of sight between the first tracking device and the tool is not blocked, determining a second position of the tool with respect to the anatomy of the patient using data from a second tracking device; and
generating, by the at least one computer processor, for display by the first head-mounted display, a second augmented reality image, the second augmented reality image comprising a virtual image of the tool aligned with a virtual image of the anatomy of the patient, and the second augmented reality image being aligned with actual patient anatomy.
15. The method of claim 14 ,
wherein the second tracking device is part of a second head-mounted device, and
wherein the second head-mounted device comprises a second head-mounted display.
16. The method of claim 14 , wherein the second tracking device is disposed in a stationary position.
17. The method of claim 14 , wherein the tracking of the tool comprises tracking a tool marker.
18. The method of claim 14 , wherein the first tracking device further comprises at least one of a light source and a camera.
19. The method of claim 14 , wherein the at least one computer processor is integrated within the first head-mounted device.
20. The method of claim 14 , wherein first head-mounted device further comprises an inertial measurement unit, and wherein the second augmented reality image is aligned with actual patient anatomy based upon data from tracking of movements of the first head-mounted device using the inertial measurement unit of the first head-mounted device.