GRAPHICAL USER INTERFACE FOR USE IN A SURGICAL NAVIGATION SYSTEM WITH A ROBOT ARM

1 .- 12 . (canceled)

13 . An apparatus, comprising:

a memory;

a processor operatively coupled to the memory, the processor configured to:

determine, based on data associated with an operative plan of a surgical procedure, a suggested position and orientation of a robot arm configured to manipulate a surgical instrument in a surgical procedure;

determine, based on tracking data associated with the robot arm, an actual position and orientation of the robot arm; and

generate a surgical navigation image including (1) a virtual object indicative of the suggested position and orientation of the robot arm, (2) a virtual representation of the robot arm, (3) a virtual representation of the surgical instrument, and (4) a virtual representation of a portion of a patient anatomy; and

a three-dimensional (3D) display system configured to display the surgical navigation image on a surface positionable between a head of an operator and a real-world environment including the patient anatomy such that the virtual representation of the portion of the patient anatomy, the virtual representation of the robot arm, and the virtual representation of the surgical instrument are collocated with real objects in the real-world environment in a field of view of the operator.

14 . The apparatus of claim 13 , wherein the 3D display system includes a 3D display and a see-through screen,

the see-through screen being positionable between the head of the operator and the real-world environment,

the 3D display configured to project the surgical navigation image onto the see-through screen.

15 . The apparatus of claim 13 , further comprising a tracking system configured to track the actual position and orientation of the robot arm,

the processer further being operatively coupled to the tracking system and being configured to receive the tracking data from the tracking system.

16 . The apparatus of claim 13 , further comprising a position system associated with the robot arm, the position system including a set of encoders located at each joint of the robot arm and configured to provide the tracking data indicative of an angle or position of each element of the robot arm.

17 . The apparatus of claim 13 , wherein the processor is further configured to adapt the surgical navigation image based on a position and orientation of the head of the operator.

18 . The apparatus of claim 13 , wherein the processor is further configured to adapt the virtual representation of the robot arm to show or hide different portions of the robot arm.

19 . The apparatus of claim 13 , wherein the processor is further configured to adapt the virtual representation of the robot arm by changing an opacity of the virtual representation of the robot arm.

20 . The apparatus of claim 13 , wherein the surgical navigation image further includes a set of images depicting different orthogonal or arbitrary planes of the portion of the patient anatomy.

21 . The apparatus of claim 13 , wherein the surgical navigation image further includes surgical guidance for placement of an implantable device using the surgical instrument within the patient anatomy.

22 . An apparatus, comprising:

a memory;

a processor operatively coupled to the memory, the processor configured to:

determine, based on tracking data associated with a robot arm configured to manipulate a surgical instrument in a surgical procedure, an actual position and orientation of the robot arm;

generate a surgical navigation image including (1) a virtual representation of the robot arm, (2) a virtual representation of the surgical instrument, and (3) a virtual representation of a portion of a patient anatomy; and

adapt the virtual representation of the robot arm by changing at least one of: an opacity of the virtual representation of the robot arm, or selectively hiding one or more portions of the virtual representation of the robot arm; and

a three-dimensional (3D) display system configured to display the surgical navigation image on a surface positionable between a head of an operator and a real-world environment including the patient anatomy such that the virtual representation of the portion of the patient anatomy, the virtual representation of the robot arm, and the virtual representation of the surgical instrument are collocated with real objects in the real-world environment in a field of view of the operator.

23 . The apparatus of claim 22 , wherein the 3D display system includes a 3D display and a see-through screen,

the see-through screen being positionable between the head of the operator and the real-world environment,

the 3D display configured to project the surgical navigation image onto the see-through screen.

24 . The apparatus of claim 22 , further comprising a tracking system configured to track the actual position and orientation of the robot arm,

the processer further being operatively coupled to the tracking system and being configured to receive the tracking data from the tracking system.

25 . The apparatus of claim 22 , further comprising a position system associated with the robot arm, the position system including a set of encoders located at each joint of the robot arm and configured to provide the tracking data indicative of an angle or position of each element of the robot arm.

26 . The apparatus of claim 22 , wherein the processor is further configured to adapt the surgical navigation image based on a position and orientation of the head of the operator.

27 . The apparatus of claim 22 , wherein the surgical navigation image further includes a set of images depicting different orthogonal or arbitrary planes of the portion of the patient anatomy.

28 . The apparatus of claim 22 , wherein the surgical navigation image further includes surgical guidance for placement of an implantable device using the surgical instrument within the patient anatomy.

29 . A method, comprising:

determining, based on data associated with an operative plan of a surgical procedure, a suggested position and orientation of a robot arm configured to manipulate a surgical instrument in a surgical procedure;

receiving, from a tracking system, tracking data associated with the robot arm;

determining, based on the tracking data associated with the robot arm, an actual position and orientation of the robot arm; and

generating a surgical navigation image including (1) a virtual object indicative of the suggested position and orientation of the robot arm, (2) a virtual representation of the robot arm, (3) a virtual representation of the surgical instrument, and (4) a virtual representation of a portion of a patient anatomy; and

displaying, using a three-dimensional (3D) display system, the surgical navigation image on a surface positionable between a head of an operator and a real-world environment including the patient anatomy such that the virtual representation of the portion of the patient anatomy, the virtual representation of the robot arm, and the virtual representation of the surgical instrument are collocated with real objects in the real-world environment in a field of view of the operator.

30 . The method of claim 29 , wherein displaying the surgical navigation image on the surface includes displaying the virtual object indicative of the suggested position and orientation of the robot arm and one or more virtual guidance cues that indicate whether the robot arm has been placed in the suggested position and orientation of the robot arm.

31 . The method of claim 29 , further comprising adapting the virtual representation of the robot arm by selectively hiding one or more portions of the virtual representation of the robot arm.

32 . The method of claim 29 , further comprising adapting the virtual representation of the robot arm by changing an opacity of the virtual representation of the robot arm.