Method of graphically tagging and recalling identified structures under visualization for robotic surgery
A system and method for augmenting an endoscopic display during a medical procedure including capturing a real-time image of a working space within a body cavity during a medical procedure. A feature of interest in the image is identified to the system using eye tracking input, and a graphical tag is displayed on the image marking the feature.
1. A method of tagging regions of interest on displayed images during a medical procedure, comprising:
positioning an endoscope in a body cavity;
positioning a surgical instrument in the body cavity;
capturing images of a surgical site within the body cavity and displaying the images on a display, the displayed images including images of the surgical instrument,
displaying a graphical pointer as an overlay on the images displayed on the display,
receiving input from an eye tracker in response to a user directing the user's gaze towards the displayed images,
in response to a user directing the user's gaze towards a first region of the surgical site as displayed in the displayed images, positioning the graphical pointer at the first region; and
in response to user selection input, displaying a graphical tag at the first region of the displayed images.
2. The method of claim 1 , wherein the method further comprises entering a tag selection mode comprising, wherein the steps of displaying the graphical pointer, positioning the graphical pointer, and displaying the graphical tag are performed in the tag selection mode.
3. The method of claim 2 , wherein the method further comprises:
exiting the tag selection mode, wherein graphical tag remains displayed at the region of interest on images of the surgical site displayed on the display.
4. The method of claim 1 , wherein the method further includes removing the graphical tag in response to user input to remove the tag.
5. The method of claim 4 , wherein the method further includes restoring the graphical tag at the region of interest in response to user input to restore the tag.
6. The method of claim 1 , wherein the selection input is sound input.
7. The method of claim 1 , wherein the selection input is received from at least one of a button, foot pedal, joystick, stylus, or touch input.
8. The method of claim 1 , further including:
mounting the surgical instrument to a first robotic manipulator arm;
causing the first robotic manipulator arm to manipulate the surgical instrument within the body cavity in in response to user manipulation of a user input handle; and
causing the first robotic manipulator to move the surgical site towards a region of the surgical site corresponding to the tagged first region.
9. The method of claim 8 , further comprising causing the first robotic manipulator to move a tip of the first surgical instrument to a location a predetermined standoff distance from the tagged first region.
10. The method of claim 9 , further including:
mounting the endoscope to a second robotic manipulator arm; and
causing the second robotic manipulator to move the endoscope towards the tagged first region.
11. The method of claim 1 , wherein the tagged first region has a depth, and wherein the method further comprises:
mounting the surgical instrument to a first robotic manipulator arm;
causing the first robotic manipulator arm to manipulate the surgical instrument within the body cavity in in response to user manipulation of a user input handle; and
preventing movement of the first instruments to a location in the body cavity that is deeper than the depth.
12. The method of claim 1 , further comprising:
detecting in real-time a change in the image during the medical procedure, the change indicative of the feature of interest becoming at least partially obscured by smoke or pooling blood; and
altering a quality of the displayed graphical tag to enhance its visibility in response to the detected change.
13. The method of claim 1 , wherein the surgical instrument is an endoscope.