Systems and methods of communicating through visual overlay for surgical medical systems
A surgical system can include a master controller for controlling one or more surgical tools. The system can also include an input on the master controller configured to change the master controller from a first mode into a second mode. The first mode can be a teleoperation mode and the second mode can be a virtual marking mode. In the virtual marking mode, a user is capable of communicating a virtual marker to other staff.
1 . A surgical system comprising:
a master controller for controlling one or more surgical tools; and
an input on the master controller configured to change the master controller from a first mode into a second mode,
wherein the first mode comprises a teleoperation mode and the second mode comprises a virtual marking mode,
wherein, in the teleoperation mode, the master controller is configured to move or actuate the one or more surgical tools, the one or more surgical tools being operable to manipulate tissue, and, in the virtual marking mode, the master controller is configured to generate a virtual marker on one or more displays, the virtual marker being generated via a user manipulation of at least one of the one or more surgical tools.
2 . A surgical system comprising:
a master controller for controlling one or more surgical tools;
a viewer on the master controller configured to display a representation of a surgical site;
a screen separate from the master controller configured to display the representation of the surgical site; and
an input on the master controller configured to change the master controller from a teleoperation mode into a virtual marking mode,
wherein, in the teleoperation mode, the master controller is configured to move or actuate the one or more surgical tools, and, in the virtual marking mode, the master controller is configured to generate a virtual marker overlayed on the representation of the surgical site in the screen separate from the master controller,
wherein in the virtual marking mode, the screen separate from the master controller is operable to one or both of (i) modify the virtual marker generated at the master controller and overlayed on the representation of the surgical site to thereby provide a modified virtual marker, or (ii) generate another virtual marker overlayed on the representation of the surgical site to thereby provide a generated other virtual marker,
wherein the viewer on the master controller is further configured to display one or both of (i) the modified virtual marker, or (ii) the generated other virtual marker.
3 . The surgical system of claim 2 , wherein the input comprises an action of one or more graspers of the master controller.
4 . The surgical system of claim 3 , wherein the action comprises a double gripping of at least one of the graspers of the master controller.
5 . The surgical system of claim 1 , wherein, in the virtual marking mode, a user at the master controller is capable of communicating the virtual marker to other staff at a patient site separate from the master controller.
6 . The surgical system of claim 2 , wherein the virtual marker comprises a hand-drawn overlay positioned over a representation of a surgical site.
7 . The surgical system of claim 2 , wherein the screen separate from the master controller comprises a screen of a tower.
8 . The surgical system of claim 2 , wherein, in the teleoperation mode, a viewer of the master controller is configured to display an image captured by a camera controlled by the master controller.
9 . The surgical system of claim 1 , wherein, in the virtual marking mode, a screen of a tower separate from the master controller is configured to display the virtual marker, and the tower is configured to generate or manipulate a virtual marker displayed on a viewer of the master controller.
10 . The surgical system of claim 7 , wherein the screen of the tower is a touchscreen configured to manipulate the virtual marker generated by the master controller.
11 . The surgical system of claim 2 , wherein, in the virtual marking mode, the master controller is configured for two-way communication between users.
12 . The surgical system of claim 2 , wherein, in the teleoperation mode, one or more graspers of the master controller are configured to move or actuate the one or more surgical tools, and, in the virtual marking mode, the one or more graspers are configured to generate or position the virtual marker.
13 . The surgical system of claim 2 , wherein, in the teleoperation mode, one or more graspers of the master controller are configured to move or actuate the one or more surgical tools, and, in the virtual marking mode, a touchpad of the master controller is configured to generate or draw the virtual marker.
14 . The surgical system of claim 2 , wherein the input comprises a button on the master controller.
15 . A method of communication during surgery comprising:
displaying, at a viewer of a master controller, a representation of a surgical site;
displaying, at a screen separate from the master controller, the representation of the surgical site;
controlling, in a teleoperation mode, one or more surgical tools in response to user input at the master controller;
changing the master controller from the teleoperation mode into a virtual marking mode in response to user input at the master controller;
generating, in the virtual marking mode, a virtual marker overlayed on the representation of the surgical site at the screen separate from the master controller in response to user input at the master controller, the virtual marker being defined by a user by manipulating the user input at the master controller such that the virtual marker is a user-defined virtual marker, the user-defined virtual marker being defined in relation to an anatomical structure at the surgical site to thereby associate the user-defined virtual marker with the anatomical structure; and
fixing the user-defined virtual marker to a point in the representation of the surgical site, the point positioned at the anatomical structure associated with the user-defined virtual marker, wherein the user-defined virtual marker remains fixed to the point in the representation of the surgical site when a camera view changes.
16 . The method of claim 15 , further comprising displaying the user-defined virtual marker on the representation of the surgical site on the viewer of the master controller.
17 . The method of claim 15 , further comprising communicating the user-defined virtual marker on the representation of the surgical site to at least two users in two different locations.
18 . The method of claim 15 , wherein the controlling of the one or more surgical tools is based on user actuation of one or more graspers of the master controller, and wherein the generating of the user-defined virtual marker is based on user actuation of the one or more graspers of the master controller.
19 . The method of claim 15 , wherein the controlling of the one or more surgical tools is based on user actuation of one or more graspers of the master controller, and wherein the generating of the user-defined virtual marker is based on movement of a finger of the user on a touchscreen of the master controller.