Graphical user interface for a surgical navigation system and method for providing an augmented reality image during operation

1. An apparatus, comprising:

a memory;

a processor operatively coupled to the memory, the processor configured to:

determine, based on data associated with an operative plan of a surgical procedure, a suggested position and orientation of a medical device in an anatomy of a patient, the medical device configured to be used in the surgical procedure;

determine, based on tracking data associated with the medical device, an actual position and orientation of the medical device when the medical device is partially inserted into the patient and includes a hidden portion disposed below a skin of the patient; and

generate a surgical navigation image including a three-dimensional (3D) image including (1) a virtual object indicative of the suggested position and orientation of the medical device, (2) a virtual representation of the medical device including a part corresponding to the hidden portion of the medical device, and (3) a virtual representation of a portion of the anatomy of the patient; and

a display system configured to display the surgical navigation image on a partially transparent and partially reflective surface positionable between a head of an operator and a surgical field including the patient anatomy such that, when viewed by the operator, (1) the virtual representation of the portion of the anatomy is collocated with the portion of the anatomy and (2) the virtual representation of the medical device is overlaid on a visible portion of the medical device and has the part corresponding to the hidden portion of the medical device extending from the visible portion of the medical device.

2. The apparatus of claim 1 , wherein the virtual representation of the medical device is a first virtual representation of the medical device, and the virtual object is a second virtual representation of the medical device having the suggested position and orientation of the medical device.

3. The apparatus of claim 1 , wherein the processor is configured to generate the surgical navigation image to further include a set of images depicting different orthogonal or arbitrary planes of the portion of the anatomy.

4. The apparatus of claim 3 , wherein the display system is configured to display the surgical navigation image on the partially transparent and partially reflective surface such that the set of images is displayed at a location next to the 3D image.

5. The apparatus of claim 3 , wherein the display system is configured to adjust the location of the set of images based on a location of the 3D image.

6. The apparatus of claim 3 , wherein the display system is configured to display the surgical navigation image on the surface such that the 3D image occupies a larger area of a field of view of the operator than the set of images.

7. The apparatus of claim 3 , wherein the processor is further configured to adapt the set of images of the surgical navigation image based on changes to the actual position and orientation of the medical device.

8. The apparatus of claim 1 , wherein the processor is further configured to adapt the 3D image based on a position and orientation of the head of the operator.

9. The apparatus of claim 1 , wherein the processor is further configured to generate the surgical navigation image to include virtual guidance indicating whether a placement of the medical device is correct based on the suggested position and orientation of the medical device.

10. The apparatus of claim 9 , wherein the virtual guidance includes an animation.

11. The apparatus of claim 1 , wherein the partially transparent and partially reflective surface is a surface of a see-through mirror, the see-through mirror being positionable at a first distance from the head of the operator and at a second distance from the surgical field, the first distance being less than the second distance.

12. The apparatus of claim 11 , wherein the display system is configured to move such that a location of the see-through mirror can move relative to the head of the operator.

13. The apparatus of claim 11 , wherein the display system includes a 3D display and an arm extending from the 3D display, the arm configured to support the see-through mirror such that the see-through mirror can be positioned between the head of the operator and the surgical field when the 3D display is positioned above the head of the operator.

14. The apparatus of claim 1 , wherein the processor is further configured to:

determine, based on tracking data associated with the head of the operator, a position and orientation of the head of the operator,

the processor configured to generate the surgical navigation image according to a perspective of the operator based on the position and orientation of the head of the operator.

15. A method, comprising:

determining, based on data associated with an operative plan of a surgical procedure, a suggested position and a suggested orientation of a medical device in an anatomy of a patient, the medical device configured to be used in the surgical procedure;

receiving, from a tracking system, tracking data associated with the medical device when the medical device is partially inserted into the patient and includes a hidden portion disposed below a skin of the patient;

determining, based on the tracking data associated with the medical device, an actual position and orientation of the medical device;

generating a three-dimensional (3D) image including (1) a first virtual guidance clue for indicating the suggested position of the medical device, (2) a second virtual guidance clue for indicating the suggested orientation of the medical device, and (3) a virtual representation of the medical device associated with the actual position and orientation of the medical device, the virtual representation including a part corresponding to the hidden portion of the medical device; and

displaying, on a partially transparent and partially reflective surface, the 3D image in a field of view of an operator such that (1) the virtual representation of the medical device is overlaid on a visible portion of the medical device in the field of view and has the part corresponding to the hidden portion of the medical device extending from the visible portion of the medical device and (2) the first and second virtual guidance clues show the operator whether the medical device has been placed in the suggested position and the suggested orientation.

16. The method of claim 15 , wherein the 3D image further includes a virtual representation of a portion of the anatomy of the patient, and the 3D image is displayed in the field of view further such that the virtual representation of the portion of the anatomy is collocated with the portion of the anatomy.

17. The method of claim 15 , further comprising:

generating a set of images depicting different orthogonal or arbitrary planes of the portion of the anatomy; and

displaying the set of images in the field of view of the operator such that the set of images is shown next to the 3D image.

18. The method of claim 17 , wherein at least one of the set of images includes one or more additional guidance clues associated with the suggested position or the suggested orientation of the medical device.

19. The method of claim 17 , further comprising adapting the sets of images based on changes to the actual position and orientation of the medical device.

20. The method of claim 15 , further comprising:

receiving, from the tracking system, tracking data associated with a head of the operator;

determining, based on the tracking data associated with the head of the operator, a position and orientation of the head of the operator; and

adapting the 3D image based on the position and orientation of the head of the operator.