Stereoscopic visualization camera and integrated robotics platform
A robotic imaging apparatus is disclosed. A robotic imaging apparatus includes a robotic arm, a stereoscopic camera, and a sensor positioned between the robotic arm and the stereoscopic camera. The sensor transmits output data that is indicative of translational and rotational force imparted on the stereoscopic camera by an operator. The robotic imaging apparatus also includes a processor that is configured to determine a movement sequence for the robotic arm based on a current position of the robotic arm and the output data from the sensor and to cause at least one of the joints of the robotic arm to rotate based on the determined movement sequence via one or more motor control signals provided to the at least one joint. The rotation of the at least one joint provides power-assisted movement of the robotic arm based on the detected translational and rotational forces imparted by the operator.
1. A robotic imaging apparatus comprising:
a robotic arm including
a first end for connection to a secure structure,
a second end including a coupling interface, and
a plurality of joints and links connecting the first end to the second end, each joint including a motor configured to rotate the joint around an axis and a joint sensor configured to transmit a position of the respective joint;
an imaging device connected to the robotic arm at the coupling interface, the imaging device configured to record images of a target surgical site;
a sensor positioned at the coupling interface and configured to detect and transmit force and/or torque output data that is indicative of force and/or torque imparted on the imaging device by an operator; and
at least one processor communicatively coupled to the sensor and the robotic arm, the at least one processor configured to:
receive the force and/or torque output data from the sensor,
convert the force and/or torque output data into translational and rotational vectors,
determine, using kinematics, a movement sequence for the robotic arm based on a current position of the robotic arm and the translational and rotational vectors, the movement sequence specifying a rotation direction, a speed, and a duration of movement for at least some of the joints of the robotic arm, and
cause at least one of the joints of the robotic arm to rotate based on the determined movement sequence via one or more motor control signals provided to the at least one joint.
2. The apparatus of claim 1 , wherein the processor is configured to:
determine a least one scale factor based on at least one of the current position of the robotic arm or a future position of the robotic arm based on the movement sequence; and
apply the scale factor to at least one joint speed of the movement sequence.
3. The apparatus of claim 2 , wherein the at least one scale factor is configured based on a distance of the robotic arm or the imaging device from a virtual boundary, and
wherein the at least one scale factor decreases to a value of ‘0’ as the virtual boundary is approached.
4. The apparatus of claim 3 , wherein the virtual boundary corresponds to at least one of a patient, a medical instrument, or operating room staff.
5. The apparatus of claim 3 , wherein the processor is configured to cause a display device to display an icon indicative that the at least one scale factor has been applied to the movement sequence.
6. The apparatus of claim 1 , wherein the processor is configured to:
determine a least one scale factor based on joint angles between joints of the robotic arm or joint limits; and
apply the scale factor to at least one joint speed of the movement sequence.
7. The apparatus of claim 1 , wherein the processor is configured to:
provide gravity compensation for the force and/or torque output data; and
provide force-application compensation for the force and/or torque output data to compensate for an offset between a location of the sensor and a location of the imaging device upon which the force and/or torque is imparted by the operator.
8. The apparatus of claim 1 , wherein the processor is configured to:
determine or identify joint singularities for the plurality of joints of the robotic arm for control of hysteresis and backlash; and
determine the movement sequence based on the kinematics while avoiding robotic arm movement through the joint singularities.
9. The apparatus of claim 1 , further comprising a coupling plate with a first end configured to connect to the coupling interface of the robotic arm and a second end including a second coupling interface configured to connect to the stereoscopic camera,
wherein the coupling plate includes at least one joint including a joint sensor configured to transmit a position of the respective joint and a motor that is controllable by the at least one processor according to the movement sequence, and
wherein the sensor is located at the coupling interface or the second coupling interface.
10. The apparatus of claim 9 , wherein the robotic arm includes at least four joints and the coupling plate includes at least two joints.
11. The apparatus of claim 1 , wherein the processor is configured to cause at least one of the joints of the robotic arm to rotate by transmitting one or more command signals to the motor of the respective joint indicative of the rotation direction, the speed, and the duration of movement as specified by the movement sequence.
12. The apparatus of claim 1 , wherein the processor is configured to compare images recorded by the imaging device as the robotic arm is being moved during the movement sequence to confirm the robotic arm is being moved as determined during the movement sequence.
13. The apparatus of claim 1 , wherein the kinematics includes at least one of inverse kinematics or Jacobean kinematics.
14. A robotic imaging apparatus comprising:
a robotic arm including
a first end for connection to a secure structure,
a second end including a coupling interface, and
a plurality of joints and links connecting the first end to the second end, each joint including a motor configured to rotate the joint around an axis and a joint sensor configured to transmit a position of the respective joint;
an imaging device connected to the robotic arm at the coupling interface, the imaging device configured to record images of a target surgical site;
a sensor positioned at the coupling interface and configured to detect and transmit force and/or torque output data that is indicative of force and/or torque imparted on the imaging device by an operator; and
at least one processor communicatively coupled to the sensor and the robotic arm, the at least one processor configured to:
receive the force and/or torque output data from the sensor,
convert the force and/or torque output data into translational and rotational vectors,
determine, using kinematics, a movement sequence for the robotic arm based on a current position of the robotic arm and the translational and rotational vectors, the movement sequence specifying a rotation direction, a speed, and a duration of movement for at least some of the joints of the robotic arm,
cause at least one of the joints of the robotic arm to rotate based on the determined movement sequence via one or more motor control signals provided to the at least one joint,
determine a least one scale factor based on at least one of the current position of the robotic arm or a future position of the robotic arm based on the movement sequence, and
apply the scale factor to at least one joint speed of the movement sequence.
15. The apparatus of claim 14 , wherein the at least one scale factor is configured based on a distance of the robotic arm or the imaging device from a virtual boundary, and
wherein the at least one scale factor decreases to a value of ‘0’ as the virtual boundary is approached.
16. The apparatus of claim 15 , wherein the virtual boundary corresponds to at least one of a patient, a medical instrument, or operating room staff.
17. The apparatus of claim 15 , wherein the processor is configured to cause a display device to display an icon indicative that the at least one scale factor has been applied to the movement sequence.
18. The apparatus of claim 14 , wherein the at least one scale factor is determined based on joint angles between joints of the robotic arm or joint limits.
19. A robotic imaging apparatus comprising:
a robotic arm including
a first end for connection to a secure structure,
a second end including a coupling interface, and
a plurality of joints and links connecting the first end to the second end, each joint including a motor configured to rotate the joint around an axis and a joint sensor configured to transmit a position of the respective joint;
an imaging device connected to the robotic arm at the coupling interface, the imaging device configured to record images of a target surgical site;
a sensor positioned at the coupling interface and configured to detect and transmit force and/or torque output data that is indicative of force and/or torque imparted on the imaging device by an operator;
at least one processor communicatively coupled to the sensor and the robotic arm, the at least one processor configured to:
receive the force and/or torque output data from the sensor,
convert the force and/or torque output data into translational and rotational vectors,
determine, using kinematics, a movement sequence for the robotic arm based on a current position of the robotic arm and the translational and rotational vectors, the movement sequence specifying a rotation direction, a speed, and a duration of movement for at least some of the joints of the robotic arm, and
cause at least one of the joints of the robotic arm to rotate based on the determined movement sequence via one or more motor control signals provided to the at least one joint; and
a coupling plate with a first end configured to connect to the coupling interface of the robotic arm and a second end including a second coupling interface configured to connect to the stereoscopic camera, wherein the coupling plate includes at least one joint including a joint sensor configured to transmit a position of the respective joint and a motor that is controllable by the at least one processor according to the movement sequence, and wherein the sensor is located at the coupling interface or the second coupling interface.
20. The apparatus of claim 19 , wherein the robotic arm includes at least four joints and the coupling plate includes at least two joints.