Displaying a virtual model of a planned instrument attachment to ensure correct selection of physical instrument attachment
A virtual model a planned instrument attachment can be provided to ensure correct selection of a physical instrument attachment. An XR headset controller can generate a shape and a pose of the virtual model of the planned instrument attachment based on predetermined information associated with the planned instrument attachment and based on a pose of an instrument relative to the XR headset. An XR headset can display the virtual model on a see-through display screen of the XR headset that is configured to allow at least a portion of a real-world scene to pass therethrough.
1. A surgical system comprising:
an extended reality (“XR”) headset configured to be worn by a user during a surgical procedure and including a see-through display screen configured to display a virtual model of a planned instrument attachment and to allow at least a portion of a real-world scene to pass therethrough for viewing by the user, the real-world scene including an instrument; and
an XR headset controller configured to generate a shape and a pose of the virtual model of the planned instrument attachment based on predetermined information associated with the planned instrument attachment and based on a pose of the instrument relative to the XR headset.
2. The surgical system of claim 1 , the XR headset controller being further configured to determine the pose of the virtual model for the planned instrument attachment to appear attached to a location on the instrument when viewed through the see-through display screen.
3. The surgical system of claim 2 , the real-world scene further includes a physical instrument attachment attached to the location on the instrument.
4. The surgical system of claim 3 , the XR headset controller being further configured to generate the virtual model of the planned instrument attachment to have partially transparent graphical features that at least one of which is posed to substantially align with at least one corresponding feature of the physical instrument attachment when the virtual model of the planned instrument attachment is overlaid on the physical instrument attachment while viewed through the see-through display.
5. The surgical system of claim 1 , the XR headset controller being further configured to generate the virtual model of the planned instrument attachment to have opaque graphical features that at least one of which is posed to substantially align with at least one corresponding feature of a physical instrument attachment when the virtual model of the planned instrument attachment is overlaid on the physical instrument attachment while viewed through the see-through display.
6. The surgical system of claim 1 , the XR headset controller being further configured to generate the virtual model of the planned instrument attachment to have opaque graphical features that at least one of which is posed to substantially align with at least one corresponding feature of the physical instrument attachment when the virtual model of the planned instrument attachment is positioned behind the physical instrument attachment while viewed through the see-through display.
7. The surgical system of claim 1 , the XR headset controller being further configured to selectively render graphical features of the virtual model of the planned instrument attachment as one of partially transparent and opaque based on user input.
8. The surgical system of claim 1 , the XR headset controller being further configured to selectively determine the pose of the virtual model of the planned instrument attachment to be displayed as one of an overlay on the physical instrument attachment, as behind the physical instrument attachment, and apart from the physical instrument attachment based on user input.
9. The surgical system of claim 1 , the XR headset controller being further configured to scale the shape of the virtual model of the planned instrument attachment based on a distance determined from the pose of the physical instrument attachment relative to the XR headset.
10. The surgical system of claim 1 , the real-world scene further includes a physical instrument attachment, and
the XR headset controller being further configured to generate the virtual model of the planned instrument attachment to have graphical features representing at least one of a pedicle screw or an expandable cage.
11. The surgical system of claim 1 , the XR headset controller being further configured to generate the virtual model of the planned instrument attachment to have graphical features representing an instrument extender.
12. The surgical system of claim 1 , the XR headset controller being further configured to generate the virtual model to include virtual text indicating a characteristic of the planned instrument attachment, based on the predetermined information associated with the planned instrument attachment.
13. The surgical system of claim 1 , further comprising:
a tracking system configured to determine the pose of the instrument relative to the XR headset;
a surgical robot that includes:
a robot base,
a robot arm connected to the robot base and configured to position an end effector which is configured to guide movement of the instrument,
at least one motor operatively connected to move the robot arm relative to the robot base; and
a navigation controller configured to:
determine a target pose for the instrument based on a surgical plan defining where a surgical procedure is to be performed using the instrument on an anatomical structure and defining the planned instrument attachment to be used during the surgical procedure, and
generate steering information based on the target pose for the instrument and the pose of the instrument, the steering information indicating where the instrument is to be moved under control of the at least one motor.
14. A surgical system comprising:
an extended reality (“XR”) headset configured to be worn by a user during a surgical procedure and including a see-through display screen configured to display a virtual model of a planned instrument attachment and to allow at least a portion of a real-world scene to pass therethrough for viewing by the user, the real-world scene including a physical instrument attachment; and
an XR headset controller configured to generate a shape and a pose of the virtual model of the planned instrument attachment based on predetermined information of the planned instrument attachment and based on a pose of the physical instrument attachment relative to the XR headset.
15. The surgical system of claim 14 , the XR headset controller being further configured to generate the pose of the virtual model of the planned instrument attachment at a predetermined pose relative to the pose of the physical instrument attachment.
16. The surgical system of claim 15 , the XR headset controller being further configured to adjust an orientation of the virtual model of the planned instrument attachment relative to the XR headset to match an orientation of the physical instrument attachment.
17. The surgical system of claim 15 , the XR headset controller being further configured to scale the shape of the virtual model of the planned instrument attachment based on a distance determined from the pose of the physical instrument attachment relative to the XR headset.
18. The surgical system of claim 15 , the XR headset controller being further configured to selectively determine the pose of the virtual model of the planned instrument attachment to be displayed as one of an overlay on the physical instrument attachment, as behind the physical instrument attachment, and apart from the physical instrument attachment based on user input.
19. The surgical system of claim 14 , the surgical system further comprising a tracking system configured to determine the pose of the physical instrument attachment relative to the XR headset,
wherein the XR headset controller is further configured to generate the virtual model to include virtual text indicating a characteristic of the planned instrument attachment, based on the predetermined information associated with the planned instrument attachment.
20. A method of providing a virtual model of a planned instrument attachment to ensure correct selection of a physical instrument attachment, the method comprising:
determining a pose of an instrument relative to an XR headset;
generating a shape and a pose of the virtual model of the planned instrument attachment based on predetermined information associated with the planned instrument attachment and based on the pose of the instrument relative to the XR headset; and
displaying the virtual model on a see-through display screen of the XR headset that is configured to allow at least a portion of a real-world scene to pass therethrough.