Handheld user interface device for a surgical robot
Disclosed herein is a mobile interface device to control a robotically-assisted surgical system. The mobile interface device provides a surgeon with a direct view of the surgical robotic system and allows a surgeon to easily and intuitively select, control, or manipulate various target components of the surgical robotic system. The mobile interface device may capture a live image of the surgical robotic system to automatically identify a robotic arm that appears in the center of the captured live image as the target component selected by the surgeon. Based on the current pose or position of the target component, the mobile interface device may generate a list of target poses and control options. The surgeon may select a control option to select a target pose and to manipulate the target component to command a robotically-assisted movement of the selected robotic arm from the current pose to the target pose.
1. A mobile interface device for a surgical robotic system, comprising:
a camera configured to capture a live image of a portion of the surgical robotic system;
a processor configured to:
process the captured live image to identify a target component of the surgical robotic system to be controlled using the mobile interface device; and
determine a current pose of the target component;
a display configured to:
display the captured live image; and
display one or more target poses of the target component; and
one or more control interfaces configured to:
allow a user to select one of the target poses; and
generate an input command to drive the target component from the current pose to the selected target pose.
2. The mobile interface of claim 1 , wherein the display is further configured to present information to visually indicate the one or more target poses of the target component to the user.
3. The mobile interface device of claim 1 , wherein the processor is further configured to determine a plurality of command options for controlling the target component and the display is further configured to display the plurality of command options for the user to control the target component.
4. The mobile interface device of claim 1 , wherein the processor is further configured to process the captured live image to identify a second component of the surgical robotic system, and the display is further configured to display information to indicate a current pose of the second component as one of the target poses of the target component.
5. The mobile interface device of claim 1 , wherein the processor is further configured to determine a trajectory to drive the target component from the current pose to the selected target pose.
6. The mobile interface device of claim 1 , further comprising one or more sensors configured to measure a three-dimensional position relative to the interface device of a plurality of surface points of the surgical robotic system captured in the live image, and wherein the processor is further configured to process the plurality of measured three-dimensional positions of the surface points of the surgical robotic system captured in the live image to identify the target component.
7. The mobile interface device of claim 1 , wherein the camera comprises a 3-dimensional camera configured to scan a patient on an operating table of the surgical robotic system during a pre-operative setup, wherein the processor is further configured to process scanned information of the patient to determine a patient-specific movement of the target component.
8. The mobile interface device of claim 1 , wherein the one or more control interfaces is further configured to generate a second input command to freeze a movement of the target component when the target component moves off the captured live image.
9. The mobile interface device of claim 1 , wherein the display is further configured to display a visual indication of the target component to allow the user to confirm the target component.
10. The mobile interface device of claim 1 , wherein the display is a touch-screen display and wherein the touch-screen display is configured to allow the user to select a current pose of a second component of the surgical robotic system in the captured live image as a desired target pose of the target component.
11. A method for controlling a surgical robotic system, comprising:
capturing, using a mobile device, a live image of a portion of the surgical robotic system;
identifying, by the mobile device or by a user, a target component of the surgical robotic system from the captured live image;
displaying, by the mobile device, the captured live image and one or more target poses of the target component;
receiving, by the mobile device from the user, a selected target pose from the one or more target poses displayed; and
generating, by the mobile device, an input command to drive the target component from a current pose to the selected target pose.
12. The method of claim 11 , wherein the displaying, by the mobile device, the one or more target poses of the target component comprises presenting information about the one or more target poses.
13. The method of claim 11 , further comprising:
generating, by the mobile device, a plurality of command options for controlling the target component; and
displaying, by the mobile device, the plurality of command options for the user to control the target component.
14. The method of claim 11 , further comprising:
identifying, by the mobile device, a second component of the surgical robotic system from the captured live image; and
displaying, by the mobile device, information to indicate a current pose of the second component as one of the target poses of the target component.
15. The method of claim 11 , further comprising:
determining, by the mobile device, a trajectory to drive the target component from the current pose to the selected target pose.
16. The method of claim 11 , further comprising:
scanning a patient on an operating table of the surgical robotic system during a pre-operative setup to obtain three-dimensional information of the patient; and
processing the three-dimensional information of the patient to determine a patient-specific movement of the target component.
17. The method of claim 11 , further comprising:
generating, by the mobile device, a second input command to freeze a movement of the target component when the target component moves off the captured live image.
18. The method of claim 11 , further comprising:
displaying, by the mobile device, a visual indication of the target component to allow the user to confirm the target component.
19. A surgical robotic system, comprising:
an operating table;
a plurality of robotic arms coupled to the operating table;
a mobile interface device configured to:
capture a live image of the operating table or the plurality of robotic arms;
display a rendered image of the operating table or the plurality of robotic arms from the live image;
receive user input to control the operating table or one of the robotic arms; and
generate an input command to control the operating table or one of the robotic arms in response to the user input; and
a control tower communicatively coupled to the mobile interface device, the operating table and the plurality of robotic arms, wherein the control tower is configured to:
receive the input command from the mobile interface device; and
generate a control command to manipulate the operating table or one of the robotic arms in response to the input command.
20. The surgical robotic system of claim 19 , wherein the mobile interface device is further configured to present a plurality of input command options to control the operating table or one of the robotic arms.