Real-time robotic-assisted dimension measurement system
Disclosed are various real-time robotic-assisted dimension measurement systems and techniques based on the collected kinematic data from one or more surgical tools within a robotic surgical system. In some embodiments, the collected kinematic data can include positions and orientations of each joint of the one or more surgical tools which are derived from positions and orientations of a set of arm joints of one or more robotic arms which the one or more surgical tools are attached to. The disclosed dimension measurement systems allow a surgeon to perform real-time length measurements of both straight-lines and curved-lines, as well as area and volume measurements on specified portions of tissues or organs by strategically placing the tips of the one or more surgical tools over the tissues or organs, while viewing the distal ends of the surgical tools in a user display.
1. A computer-implemented method for performing real-time robotic-assisted length measurements within a robotic surgical system, the method comprising:
after initiating a length measurement during a surgical procedure on a patient, calibrating measurement errors associated with using a tool-tip of a first surgical tool with a known length of a second surgical tool of the robotic surgical system, wherein the first surgical tool is attached to a robotic arm comprising a plurality of arm joints, wherein the measurement errors include an accumulated encoder error associated with a plurality of encoders of the plurality of arm joints;
obtaining a first set of kinematic data of a wrist joint of the first surgical tool corresponding to placing the tool-tip of the first surgical tool at a first location on an anatomy, wherein the first set of kinematic data are measured relative to a base reference frame;
obtaining a second set of kinematic data of the wrist joint of the first surgical tool corresponding to placing the tool-tip of the first surgical tool at a second location on the anatomy, wherein the second set of kinematic data are measured relative to the base reference frame;
transforming the first set of kinematic data into a first position vector of the tool-tip at the first location and the second set of kinematic data into a second position vector of the tool-tip at the second location; and
computing a first distance between the first position vector and the second position vector as a measured length of a straight line between the first location and the second location on the anatomy.
2. The computer-implemented method of claim 1 , wherein prior to obtaining the first set and the second set of kinematic data and prior to calibrating the measurement errors, the method further comprises receiving a first user command to initiate the length measurement on the anatomy via a user interface device (UID) of the robotic surgical system.
3. The computer-implemented method of claim 2 , wherein after initiating the length measurement, the method further comprises:
receiving a second user command indicative of whether the length measurement is a straight-line measurement or a curved-lined measurement; and
entering either a straight-line measurement mode or a curved-line measurement mode after processing the second user command.
4. The computer-implemented method of claim 1 , wherein obtaining the first set or the second set of kinematic data of the wrist joint of the first surgical tool includes:
retrieving, from a data log of the robotic surgical system, a position vector corresponding to a center point of the wrist joint; and
retrieving, from the data log, a rotation vector corresponding to an orientation of an end effector of the first surgical tool connected to the wrist joint, wherein the end effector is positioned between the wrist joint and the tool-tip.
5. The computer-implemented method of claim 4 , wherein transforming the first set or the second set of kinematic data into the first position vector or the second position vector of the first location or the second location includes:
computing a rotational displacement of the tool-tip with respect to the center point of the wrist joint based at least on the corresponding rotation vector and a length of the end effector; and
obtaining the first position vector or the second position vector of the first location or the second location by combining the first position vector or the second position vector of the center point of the wrist joint with the corresponding rotational displacement.
6. The computer-implemented method of claim 1 , wherein the plurality of arm joints includes a first arm joint and a last arm joint, and wherein obtaining the first set or the second set of kinematic data of the wrist joint includes:
for each arm joint in the plurality of arm joints, obtaining a set of kinematic data generated by an encoder of the arm joint, wherein the set of kinematic data are measured relative to a reference frame of the arm joint;
performing a series of forward transformations from the first arm joint to the last arm joint using the sets of kinematic data associated with the plurality of arm joints, wherein each forward transformation in the series of forward transformations transforms the corresponding set of kinematic data at the corresponding arm joint into a transformed set of kinematic data for the corresponding arm joint relative to the base frame; and
performing a final forward transformation which transforms the transformed set of kinematic data at the final arm joint into the first set or the second set of kinematic data of the wrist joint of the first surgical tool in the base frame.
7. The computer-implemented method of claim 1 , wherein calibrating the measurement errors with the known length of the second surgical tool includes:
taking multiple length measurements of the known length using the tool-tip of the first surgical tool to obtain a set of measurement values of the known length; and
comparing the set of measurement values against the known length to obtain a mean error of the set of measurement values.
8. The computer-implemented method of claim 7 , wherein taking a length measurement of the known length using the tool-tip of the first surgical tool includes:
obtaining a third position vector of the tool-tip at a first end-point of the known length on the second surgical tool;
obtaining a fourth position vector of the tool-tip at a second end-point of the known length on the second surgical tool; and
generating a measurement value of the known length as the distance between the third position vector and the fourth position vector.
9. The computer-implemented method of claim 1 , wherein the method further comprises using the calibrated measurement errors to offset the measured length of the straight line to improve a measurement accuracy of using the tool-tip of the first surgical tool for length measurements.
10. The computer-implemented method of claim 1 , wherein the method further comprises:
obtaining a third set of kinematic data of the wrist joint of the first surgical tool corresponding to placing the tool-tip of the first surgical tool at a third location on the anatomy, wherein the third set of kinematic data are measured relative to the base reference frame;
transforming the third set of kinematic data into a third position vector of the tool-tip at the third location;
computing a second distance between the second position vector and the third position vector as a measured length of a straight line between the second location and the third location on the anatomy; and
outputting a sum of the first distance and the first distance as an approximated length of a curved line specified by the first, the second and the third locations.
11. A processor of a robotic surgical system, the processor configured to:
after initiating a length measurement during a surgical procedure on a patient, calibrate measurement errors associated with using a tool-tip of a first surgical tool with a known length of a second surgical tool of the robotic surgical system, wherein the first surgical tool is attached to a robotic arm comprising a plurality of arm joints, wherein the measurement errors include an accumulated encoder error associated with a plurality of encoders of the plurality of arm joints;
obtain a first set of kinematic data of a wrist joint of the first surgical tool corresponding to placing the tool-tip of the first surgical tool at a first location on an anatomy, wherein the first set of kinematic data are measured relative to a base reference frame;
obtain a second set of kinematic data of the wrist joint of the first surgical tool corresponding to placing the tool-tip of the first surgical tool at a second location on the anatomy, wherein the second set of kinematic data are measured relative to the base reference frame;
transform the first set of kinematic data into a first position vector of the tool-tip at the first location and the second set of kinematic data into a second position vector of the tool-tip at the second location; and
compute a first distance between the first position vector and the second position vector as a measured length of a straight line between the first location and the second location on the anatomy.
12. The processor of claim 11 , wherein the processor is configured to receive, prior to obtaining the first set and the second set of kinematic data and prior to calibrating the measurement errors, a first user command to initiate the length measurement on the anatomy via a user interface device (UID) of the robotic surgical system.
13. The processor of claim 12 , wherein after initiating the length measurement, the processor is configured to:
receive a second user command indicative of whether the length measurement is a straight-line measurement or a curved-lined measurement; and
enter either a straight-line measurement mode or a curved-line measurement mode after processing the second user command.
14. The processor of claim 11 , wherein obtaining the first set or the second set of kinematic data of the wrist joint of the first surgical tool includes:
retrieving, from a data log of the robotic surgical system, a position vector corresponding to a center point of the wrist joint; and
retrieving, from the data log, a rotation vector corresponding to an orientation of an end effector of the first surgical tool connected to the wrist joint, wherein the end effector is positioned between the wrist joint and the tool-tip.
15. The processor of claim 11 , wherein transforming the first set or the second set of kinematic data into the first position vector or the second position vector of the first location or the second location includes:
computing a rotational displacement of the tool-tip with respect to a center point of the wrist joint based at least on a rotation vector and a length of an end effector of the surgical tool; and
obtaining the first position vector or the second position vector of the first location or the second location by combining the first position vector or the second position vector of the center point of the wrist joint with the rotational displacement.
16. The processor of claim 11 , wherein the plurality of arm joints, including a first arm joint and a last arm joint, and wherein obtaining the first set or the second set of kinematic data of the wrist joint includes:
for each arm joint in the plurality of arm joints, obtaining a set of kinematic data generated by an encoder of the arm joint, wherein the set of kinematic data are measured relative to a reference frame of the arm joint;
performing a series of forward transformations from the first arm joint to the last arm joint using the sets of kinematic data associated with the plurality of arm joints, wherein each forward transformation in the series of forward transformations transforms the corresponding set of kinematic data at the corresponding arm joint into a transformed set of kinematic data for the corresponding arm joint relative to the base frame; and
performing a final forward transformation which transforms the transformed set of kinematic data at the final arm joint into the first set or the second set of kinematic data of the wrist joint of the first surgical tool in the base frame.
17. A non-transitory machine-readable medium having instructions which when executed by at least one processor of a robotic surgical system causes the robotic surgical system to:
after initiating a length measurement during a surgical procedure on a patient, calibrate measurement errors associated with using a tool-tip of a first surgical tool with a known length of a second surgical tool of the robotic surgical system, wherein the first surgical tool is attached to a robotic arm comprising a plurality of arm joints, wherein the measurement errors include an accumulated encoder error associated with a plurality of encoders of the plurality of arm joints;
obtain a first set of kinematic data of a wrist joint of the first surgical tool corresponding to placing the tool-tip of the first surgical tool at a first location on an anatomy, wherein the first set of kinematic data are measured relative to a base reference frame;
obtain a second set of kinematic data of the wrist joint of the first surgical tool corresponding to placing the tool-tip of the first surgical tool at a second location on the anatomy, wherein the second set of kinematic data are measured relative to the base reference frame;
transform the first set of kinematic data into a first position vector of the tool-tip at the first location and the second set of kinematic data into a second position vector of the tool-tip at the second location; and
compute a first distance between the first position vector and the second position vector as a measured length of a straight line between the first location and the second location on the anatomy.
18. The non-transitory machine-readable medium of claim 17 , wherein the non-transitory machine-readable medium comprises further instructions to receive, prior to obtaining the first set and the second set of kinematic data and prior to calibrating the measurement errors, a first user command to initiate the length measurement on the anatomy via a user interface device (UID) of the robotic surgical system.