Systems and methods for surgical navigation
A mixed reality system includes a head-mounted device (HMD) having an HMD coordinate system and a camera to capture images. A localizer has a localizer coordinate system and position sensors supported by a housing to track a pose of a real object. Registration markers are configured to be recognized from the captured images of the camera of the HMD. The registration markers are disposed on the housing of the localizer in a predetermined spatial relationship with respect to the localizer coordinate system. At least one controller is configured to utilize the registration markers recognized from the captured images to register the localizer coordinate system and the HMD coordinate system.
1. A surgical system comprising:
a head-mounted device (HMD) having an HMD coordinate system and comprising a camera configured to capture images;
a localizer having a localizer coordinate system, the localizer comprising a housing and one or more position sensors supported by the housing and being configured to track a pose of a real object;
a plurality of registration markers configured to be recognized from the captured images of the camera of the HMD, wherein the registration markers are disposed on the housing of the localizer in a predetermined spatial relationship with respect to the localizer coordinate system; and
at least one controller configured to utilize the registration markers recognized from the captured images to:
obtain a first transform defining a first relationship between the HMD coordinate system and the registration markers;
obtain a second transform defining a second relationship between the registration markers and the localizer coordinate system based on the predetermined spatial relationship of the registration markers relative to the localizer coordinate system; and
combine the first transform and the second transform to register the localizer coordinate system and the HMD coordinate system.
2. The surgical system of claim 1 , wherein:
the HMD comprises a display that is configured to overlay a computer-generated image onto a real-world view; and
the at least one controller is configured to register the localizer coordinate system and the HMD coordinate system to enable the HMD to overlay the computer-generated image onto the real object tracked by the localizer.
3. The surgical system of claim 1 , wherein:
the HMD comprises a depth sensor; and
the at least one controller is configured to obtain the first transform based on poses of the registration markers recognized from the captured images and based on a distance between the HMD and the registration markers determined by the depth sensor.
4. The surgical system of claim 1 , wherein the housing of the localizer has an elongated shape and extends between a first end and an opposing second end, and wherein a first position sensor is located adjacent to the first end and a second position sensor is located adjacent to the opposing second end, and wherein the registration markers are disposed on the housing in a region between the first position sensor and the second position sensor.
5. The surgical system of claim 1 , wherein the plurality of registration markers comprises three or more registration markers and wherein each of the registration markers comprises a pattern that is unique from the other registration markers.
6. The surgical system of claim 1 , wherein:
the HMD comprises a display positionable in front of eyes of a user; and
the at least one controller is configured to instruct the display of the HMD to display a computer-generated indicator to convey information related to registration between the localizer coordinate system and the HMD coordinate system.
7. The surgical system of claim 6 , wherein the computer-generated indicator is:
displayed while the camera of the HMD is directed towards the registration markers; and
configured to be overlaid onto a real-world view of the localizer.
8. The surgical system of claim 7 , wherein the computer-generated indicator is configured to be overlaid to encircle a real-world view of the one or more position sensors of the localizer.
9. The surgical system of claim 6 , wherein the computer-generated indicator is configured to convey progress, and/or completion, of registration between the localizer coordinate system and the HMD coordinate system.
10. The surgical system of claim 1 , wherein:
the housing of the localizer is supported by an articulated arm coupled to a cart assembly;
the localizer is an optical localizer;
the one or more position sensors are one or more optical position sensors; and
an optical tracker is coupled to the real object and is configured to be detected by the one or more optical position sensors to track the pose of the real object.
11. A mixed reality system configured to be used with a surgical navigation system comprising a localizer having a localizer coordinate system, the localizer comprising a housing and one or more position sensors supported by the housing and being configured to track a pose of a real object and a plurality of registration markers disposed on the housing of the localizer in a predetermined spatial relationship with respect to the localizer coordinate system, the mixed reality system comprising:
a head-mounted device (HMD) having an HMD coordinate system and comprising a camera configured to capture images; and
at least one controller configured coupled to the HMD and being configured to:
recognize the plurality of registration markers disposed on the housing of the localizer from the captured images of the camera of the HMD; and
utilize the registration markers recognized from the captured images to:
obtain a first transform defining a first relationship between the HMD coordinate system and the registration markers;
obtain a second transform defining a second relationship between the registration markers and the localizer coordinate system based on the predetermined spatial relationship of the registration markers relative to the localizer coordinate system; and
combine the first transform and the second transform to register the localizer coordinate system and the HMD coordinate system.
12. The mixed reality system of claim 11 , wherein:
the HMD comprises a display that is positionable in front of eyes of a user and is configured to overlay a computer-generated image onto a real-world view; and
the at least one controller is configured to register the localizer coordinate system and the HMD coordinate system to enable the HMD to overlay the computer-generated image onto the real object tracked by the localizer.
13. The mixed reality system of claim 12 , wherein the real object tracked by the localizer is a bone of a patient, and wherein the computer-generated image is a virtual anatomical model overlaid onto the bone tracked by the localizer.
14. The mixed reality system of claim 13 , wherein the computer-generated image of the virtual anatomical model is updated to represent material removed from the bone by a surgical tool.
15. The mixed reality system of claim 11 , wherein:
the HMD comprises a display that is positionable in front of eyes of a user; and
the at least one controller is configured to instruct the display of the HMD to display a computer-generated indicator to convey information related to registration between the localizer coordinate system and the HMD coordinate system.
16. The mixed reality system of claim 15 , wherein the computer-generated indicator is:
displayed while the camera of the HMD is directed towards the registration markers; and
configured to be overlaid onto a real-world view of the localizer.
17. The mixed reality system of claim 16 , wherein the computer-generated indicator is configured to be overlaid to encircle a real-world view of the one or more position sensors of the localizer.
18. The mixed reality system of claim 15 , wherein the computer-generated indicator is configured to convey progress, and/or completion, of registration between the localizer coordinate system and the HMD coordinate system.
19. A head-mounted device (HMD) to be used with a surgical navigation system that includes a localizer having a localizer coordinate system, the localizer comprising a housing and one or more position sensors supported by the housing to track a pose of a real object, and registration markers disposed on the housing in a predetermined spatial relationship with respect to the localizer coordinate system, the HMD comprising:
an HMD coordinate system;
a camera configured to capture images; and
at least one controller configured to:
recognize the registration markers disposed on the housing of the localizer from the captured images of the camera; and
utilize the registration markers recognized from the captured images to:
obtain a first transform defining a first relationship between the HMD coordinate system and the registration markers;
obtain a second transform defining a second relationship between the registration markers and the localizer coordinate system based on the predetermined spatial relationship of the registration markers relative to the localizer coordinate system; and
combine the first transform and the second transform to register the localizer coordinate system and the HMD coordinate system.