Co-manipulation surgical systems having optical sensors for generating graphical displays
Co-manipulation robotic systems are described herein that may be used for assisting with laparoscopic surgical procedures. The co-manipulation robotic systems allow a surgeon to use commercially-available surgical tools while providing benefits associated with surgical robotics. Advantageously, the surgical tools may be seamlessly coupled to the robot arms using a disposable coupler while the reusable portions of the robot arm remain in a sterile drape. Further, the co-manipulation robotic system may operate in multiple modes to enhance usability and safety, while allowing the surgeon to position the instrument directly with the instrument handle and further maintain the desired position of the instrument using the robot arm.
1. A co-manipulation surgical system comprising:
a robot arm comprising a proximal end, a distal end configured to be removably coupled to a surgical instrument, a plurality of links, and a plurality of joints;
a platform configured to support the robot arm, the platform comprising a plurality of wheels configured to permit mobility of the platform;
an optical sensor mounted on the platform and configured to collect depth data;
a display mounted on the platform; and
a controller configured to permit the robot arm to move for performing surgery using the surgical instrument, the controller operatively coupled to the optical sensor and comprising instructions that, when executed by one or more processors of the controller, cause the controller to:
receive the depth data collected by the optical sensor;
generate graphical representations of an area surrounding the platform based on the depth data, the graphical representations comprising the area surrounding the platform in real-time; and
cause the display to display the graphical representations of the area surrounding the platform in real-time to guide movement of the platform by a user within an operating room.
2. The co-manipulation surgical system of claim 1 , wherein the optical sensor comprises one or more depth cameras selected from a stereo camera, a structured light camera, and/or a time-of-flight camera.
3. The co-manipulation surgical system of claim 1 , wherein the optical sensor comprises an electromagnetic, capacitive, and/or infrared proximity sensor.
4. The co-manipulation surgical system of claim 1 , wherein at least a portion of the optical sensor is at an upper portion of the platform.
5. The co-manipulation surgical system of claim 4 , wherein the optical sensor is configured to collect depth data indicative of surgical bed height and angular orientation, patient position, trocar port position and orientation, presence of the surgical instrument, surgical instrument position, movement, and orientation, motion of one or more handheld surgical instruments, surgeon movement, presence of a sterile drape, and/or attachment and detachment of the surgical instrument to the distal end of the robot arm.
6. The co-manipulation surgical system of claim 1 , wherein at least a portion of the optical sensor is at a lower portion of the platform.
7. The co-manipulation surgical system of claim 6 , wherein the optical sensor is configured to collect depth data indicative of relative position between the platform and a surgical bed, and/or one or more objects within the area surrounding the platform.
8. The co-manipulation surgical system of claim 1 , wherein the optical sensor comprises a camera at an upper portion of the platform and a proximity sensor at a lower portion of the platform.
9. The co-manipulation surgical system of claim 1 , wherein the optical sensor comprises multiple optical sensors spaced apart on the platform.
10. The co-manipulation surgical system of claim 1 , wherein the optical sensor comprises an adjustable field of view.
11. The co-manipulation surgical system of claim 1 , wherein the graphical representations generated by the controller comprise a virtual map with a top perspective of the area surrounding the platform.
12. The co-manipulation surgical system of claim 1 , wherein the graphical representations comprise one or more objects and/or one or more persons within the area surrounding the platform.
13. The co-manipulation surgical system of claim 12 , wherein the graphical representations comprise the platform and the robot arm relative to the one or more objects and/or the one or more persons within the area surrounding the platform to facilitate avoidance of collision between the platform and/or the robot arm with the one or more objects and/or the one or more persons during movement of the platform.
14. The co-manipulation surgical system of claim 12 , wherein the controller is configured to generate an alert if the depth data indicates that the platform is within a predetermined distance from the one or more objects and/or the one or more persons within the area surrounding the platform.
15. The co-manipulation surgical system of claim 14 , wherein the alert comprises a visual and/or audible alert.
16. The co-manipulation surgical system of claim 1 , wherein the controller is configured to permit the robot arm to be freely moveable responsive to movement at a handle of the surgical instrument for performing surgery using the surgical instrument.
17. The co-manipulation surgical system of claim 16 , wherein the controller is configured to record depth data collected by the optical sensor and data indicative of positions of the robot arm during performance of the surgery.
18. The co-manipulation surgical system of claim 1 , further comprising:
a brake mechanism configured to be engaged to prevent mobility of the platform; and
an actuator operatively coupled to the brake mechanism, the actuator configured to be actuated to disengage the brake mechanism to permit mobility of the platform.
19. The co-manipulation surgical system of claim 18 , wherein the controller is configured to cause the display to display the graphical representations only while the braking mechanism is disengaged.
20. The co-manipulation surgical system of claim 1 , wherein the graphical representations comprise a surgical bed within the area surrounding the platform and the controller is configured to cause the display to display a proximity between the platform and the surgical bed.
21. The co-manipulation surgical system of claim 1 , wherein the controller is configured to cause the robot arm to move relative to the platform to avoid a collision with one or more objects and/or one or more persons based on the depth data collected by the optical sensor.
22. A method for assisting with surgery, the method comprising:
providing a robot arm comprising a plurality of links, a plurality of joints, a proximal end supported by a platform comprising a plurality of wheels configured to permit mobility of the platform, and a distal end configured to be removably coupled to a surgical instrument;
collecting, by a controller operatively coupled to the robot arm, depth data from an optical sensor mounted on the platform;
generating, by the controller, graphical representations of an area surrounding the platform based on the depth data, the graphical representations comprising the area surrounding the platform in real-time; and
causing, by the controller, a display mounted on the platform to display the graphical representations of the area surrounding the platform in real-time to guide movement of the platform by a user within an operating room.
23. The method of claim 22 , wherein the optical sensor comprises one or more depth cameras selected from a stereo camera, a structured light camera, and/or a time-of-flight camera.
24. The method of claim 22 , wherein the optical sensor comprises an electromagnetic, capacitive, and/or infrared proximity sensor.
25. The method of claim 22 , wherein causing the display to display the graphical representations comprises causing, by the controller, the display to display the graphical representations only while the platform is moving to guide movement of the platform by the user within the operating room.
26. The method of claim 22 , wherein the graphical representations comprise the platform relative to an object and/or person within the area surrounding the platform.
27. The method of claim 26 , wherein the graphical representations comprise the robot arm relative to the object and/or the person within the area surrounding the platform to facilitate avoidance of collision between the platform and/or the robot arm with the object and/or the person during movement of the platform.
28. The method of claim 26 , further comprising generating, by the controller, an alert if the depth data indicates that the platform is within a predetermined distance from the object and/or the person within the area surrounding the platform.
29. The method of claim 22 , wherein the plurality of wheels comprise a brake mechanism configured to be engaged to prevent mobility of the platform, the method further comprising actuating an actuator operatively coupled to the brake mechanism to disengage the brake mechanism to permit mobility of the platform.
30. The method of claim 22 , further comprising permitting, by the controller, the robot arm to be freely moveable responsive to movement at a handle of the surgical instrument for performing surgery using the surgical instrument.