Collaborative surgical robotic platform for autonomous task execution

1 . A method for controlling a robotic device to perform a tissue modification task, the method comprising, by a processor:

performing a coarse registration, wherein a user positions and orients a camera of a vision system and a registration tool tip of a surgical tool when in contact with tissue such that a real-time image of the tissue as captured by the camera is roughly aligned to a reference image of a contacted tissue;

identifying, using the robotic device in a hand-guidance mode, a plurality of registration points on a tissue surface, and mapping the plurality of registration points to a patient's image data; and

transforming a task plan relative to the patient's image data to a second coordinate frame of the robotic device;

controlling, using the task plan, the robotic device to autonomously or semi-autonomously execute a tissue modification task by:

incrementally moving the surgical tool from a pre-entry position towards an entry point,

measuring force applicable on the surgical tool at each incremental position until the surgical tool reaches a first datum point,

identifying the first datum point as an entry point of the surgical tool into the tissue, where a force applicable on the surgical tool at the first datum point is greater than a first threshold value,

activating, while being positioned at the first datum point, the surgical tool to perform the tissue modification task along a planned trajectory,

determining, based on at least the force measurements on the surgical tool or a position of the tool along the planned trajectory, whether the surgical tool has reached an end of the planned trajectory,

in response to determining that the surgical tool has reached the end of the planned trajectory, retracting the surgical tool from the tissue.

2 . The method of claim 1 , further comprising displaying, on a display device, a position of the surgical tool in relation to the patient's image data and the planned trajectory in real-time.

3 . The method of claim 1 , further comprising controlling the robotic device to position the surgical tool at the pre-entry position that is within a threshold distance of the entry point and at an angle of entry into the tissue.

4 . The method of claim 1 , further comprising continuously monitoring a force applicable on the surgical tool while performing the tissue modification task by subtracting a reference force from a measured force, the reference force comprising a gravitation force.

5 . The method of claim 4 , further comprising, upon activation of the surgical tool:

controlling the robotic device to position the surgical tool at a first incremental position along the planned trajectory to perform the tissue modification task;

determining, at the first incremental position, whether the force applicable has become less than a second threshold value; and

in response to determining that the force applicable has become less than the second threshold value, controlling the robotic device to position the surgical tool at a second incremental position along the planned trajectory to perform the tissue modification task.

6 . The method of claim 5 , further comprising, in response to determining that the force applicable has not become less than the second threshold value, continuing the monitoring of the force applicable on the surgical tool while performing the tissue modification task until expiration of a first time period indicative of a risk.

7 . The method of claim 6 , further comprising transmitting an alert to a user when the force applicable exceeds a third threshold value.

8 . The method of claim 7 , further comprising determining the first threshold value, the second threshold value, and the third threshold value based on at least an estimated stiffness of the tissue.

9 . The method of claim 1 , further comprising controlling the robotic device to abort the tissue modification task in response to determining that a search limit has been exceeded prior to identification of the first datum point, the search limit comprising at least one of the following: a time, a distance, or a number of attempts.

10 . The method of claim 1 , wherein identifying the plurality of registration points on the tissue surface comprises:

attaching a registration tool to the robotic device;

monitoring applicable force on the registration tool while the registration tool is contacted with a plurality of points over the tissue surface in the hand-guidance mode comprising subtracting a reference force from a measured force, the reference force comprising a gravitation force; and

identifying one or more positions of the registration tool as registration points when the monitored applicable force is greater than a contact threshold value.

11 . The method of claim 10 , further comprising performing, using one or more of the plurality of registration points, segmentation of three-dimensional surface information of the tissue collected by a vision system to identify a second plurality of registration points proximate to the one or more of the plurality of registration points.

12 . The method of claim 1 further comprising receiving the task plan, relative to the patient's image data, the task plan comprising information relating to the entry point of the surgical tool into the tissue and the planned trajectory of the surgical tool inside the tissue for performing the tissue modification task;

determining a spatial relationship between a first coordinate frame of the task plan and the second coordinate frame of the robotic device.

13 . The method of claim 2 , wherein receiving the task plan comprises receiving the task plan from a user by user-guided contact of the surgical tool with a patient's anatomy and at least one user-specified plan parameter.

14 . A system for controlling a robotic device to perform a tissue modification task, the system comprising a robotic device comprising a surgical tool; a processor; and a non-transitory computer readable medium comprising one or more programming instructions that when executed by the processor will cause the processor to:

perform a coarse registration, wherein a coarse registration step comprises a user positioning and orienting a camera of a vision system and registration tool tip when in contact with an anatomy such that a real-time image of the anatomy as captured by the camera is roughly aligned to a reference image of the contacted anatomy,

identify, using the robotic device in a hand-guidance mode, a plurality of registration points on the tissue surface, and map the plurality of registration points to a patient's image data; and transform the task plan relative to the patient's image data to a second coordinate frame of the robotic device;

control, using the task plan, the robotic device to autonomously or semi-autonomously execute the tissue modification task by:

incrementally moving the surgical tool from a pre-entry position towards the entry point until the surgical tool reaches a first datum point; and,

measuring force applicable on the surgical tool at each incremental position until the surgical tool reaches the first datum point,

identifying the first datum point as the entry point of the surgical tool into the tissue, where a force applicable on the surgical tool at the first datum point is greater than a first threshold value,

activating, while being positioned at the first datum point, the surgical tool to perform the tissue modification task along a planned trajectory,

determining, based on force measurements on the surgical tool or a position of the tool along the planned trajectory, whether the surgical tool has reached an end of the planned trajectory,

in response to determining that the surgical tool has reached the end of the planned trajectory, retracting the surgical tool from the tissue.

15 . The system of claim 14 , further comprising programming instructions that when executed by the processor will cause the processor to display, on a display device, a position of the surgical tool in relation to the patient's image data and the planned trajectory in real-time.

16 . The system of claim 14 , further comprising programming instructions that when executed by the processor will cause the processor to control the robotic device to execute the tissue modification task by positioning the surgical tool at the pre-entry position that is within a threshold distance of the entry point and at an angle of entry into the tissue.

17 . The system of claim 14 , further comprising programming instructions that when executed by the processor will cause the processor to continuously monitor a force applicable on the surgical tool while performing the tissue modification task by subtracting a reference force from a measured force, the reference force comprising a gravitation force.

18 . The system of claim 17 , further comprising programming instructions that when executed by the processor will cause the processor to, upon activation of the surgical tool:

control the robotic device to position the surgical tool at a first incremental position along the planned trajectory to perform the tissue modification task;

determine, at the first incremental position, whether the force applicable has become less than a second threshold value; and

in response to determining that the force applicable has become less than the second threshold value, control the robotic device to position the surgical tool at a second incremental position along the planned trajectory to perform the tissue modification task.

19 . The system of claim 18 , further comprising programming instructions that when executed by the processor will cause the processor to, in response to determining that the force applicable has not become less than the second threshold value, continue the monitoring of the force applicable on the surgical tool while performing the tissue modification task until expiration of a first time period indicative of a risk.

20 . The system of claim 19 , further comprising further comprising programming instructions that when executed by the processor will cause the processor to transmit an alert to a user when the force applicable exceeds a third threshold value.

21 . The system of claim 20 , further comprising further comprising programming instructions that when executed by the processor will cause the processor to determine the first threshold value, the second threshold value, and the third threshold value based on at least an estimated stiffness of the tissue.

22 . The system of claim 14 , further comprising further comprising programming instructions that when executed by the processor will cause the processor to control the robotic device to abort the tissue modification task in respond to determining that a search limit has been exceeded prior to identification of the datum point, the search limit comprising at least one of the following: a time, a distance, or a number of attempts.

23 . The system of claim 14 , wherein the programming instructions that when executed by the processor will cause the processor to identify the plurality of registration points on the tissue surface comprise programming instructions that when executed by the processor will cause the processor to:

attach a registration tool to the robotic device;

monitor applicable force on the registration tool while the registration tool is contacted with a plurality of points over the tissue surface in the hand-guidance mode and subtract a reference force from a measured force, the reference force comprising a gravitation force; and

identify one or more positions of the registration tool as registration points when the monitored applicable force is greater than a contact threshold value.

24 . The system of claim 23 , further comprising programming instructions that when executed by the processor will cause the processor to perform, using one or more of the plurality of registration points, segmentation of three-dimensional surface information a camera image of the tissue collected by a vision system to identify a second plurality of registration points proximate to the one or more of the plurality of registration points.

25 . The system of claim 14 , wherein the programming instructions that when executed by the processor will cause the processor to receive the task plan comprise programming instructions that when executed by the processor will cause the processor to receive the task plan from a user by user-guided contact of the surgical tool with a patient's anatomy and at least one user-specified plan parameter.

26 . The system of claim 14 , further comprising programming instructions that when executed by the processor will cause the processor to receive the task plan, relative to a patient's image data, the task plan comprising information relating to an entry point of the surgical tool into a tissue and a planned trajectory of the surgical tool inside the tissue for performing the tissue modification task, and determine a spatial relationship between a first coordinate frame of a task plan and a second coordinate frame of the robotic device.

27 . A method for controlling a robotic device to perform a tissue modification task, the method comprising, by a processor:

receiving a task plan, relative to a patient's image data, the task plan comprising information relating to an entry point of a surgical tool into a tissue and a planned trajectory of the surgical tool inside the tissue for performing the tissue modification task;

determining a spatial relationship between a first coordinate frame of a task plan and a second coordinate frame of the robotic device by:

performing a coarse registration step, wherein a user positions and orients a camera of a vision system and a registration tool tip when in contact with the tissue such that a real-time image of the tissue as captured by the camera is roughly aligned to a reference image of the contacted tissue;

identifying, using the robotic device in a hand-guidance mode, a plurality of registration points on a tissue surface, and mapping the plurality of registration points to a patient's image data; and

transforming the task plan relative to the patient's image data to the second coordinate frame of the robotic device;

controlling, using the task plan, the robotic device to autonomously or semi-autonomously execute the tissue modification task by:

incrementally moving the surgical tool from a pre-entry position towards the entry point,

measuring force applicable on the surgical tool at each incremental position until the surgical tool reaches a first datum point,

identifying the first datum point as the entry point of the surgical tool into the tissue, where a force applicable on the surgical tool at the first datum point is greater than a first threshold value,

activating, while being positioned at the first datum point, the surgical tool to perform the tissue modification task along the planned trajectory,

determining, based on at least the force measurements on the surgical tool or a position of the tool along the planned trajectory, whether the surgical tool has reached an end of the planned trajectory,

in response to determining that the surgical tool has reached the end of the planned trajectory, retracting the surgical tool from the tissue.

28 . A system for controlling a robotic device to perform a tissue modification task, the system comprising

a robotic device comprising a surgical tool;

a processor; and

a non-transitory computer readable medium comprising one or more programming instructions that when executed by the processor will cause the processor to:

receive a task plan, relative to a patient's image data, the task plan comprising information relating to an entry point of the surgical tool into a tissue and a planned trajectory of the surgical tool inside the tissue for performing the tissue modification task;

determine a spatial relationship between a first coordinate frame of a task plan and a second coordinate frame of the robotic device wherein determining the spatial relationship further comprises programming instructions that when executed by the processor cause the processor to;

perform a coarse registration, wherein a coarse registration step comprises a user positioning and orienting a camera of a vision system and registration tool tip when in contact with an anatomy such that a real-time image of the anatomy as captured by the camera is roughly aligned to a reference image of the contacted anatomy;

identify, using the robotic device in a hand-guidance mode, a plurality of registration points on the tissue surface, and

map the plurality of registration points to a patient's image data; and transform the task plan relative to the patient's image data to the second coordinate frame of the robotic device;

control, using the task plan, the robotic device to autonomously or semi-autonomously execute the tissue modification task by:

incrementally moving the surgical tool from a pre-entry position towards the entry point until the surgical tool reaches a first datum point; and,

measuring force applicable on the surgical tool at each incremental position until the surgical tool reaches the first datum point,

identifying the first datum point as the entry point of the surgical tool into the tissue, where a force applicable on the surgical tool at the first datum point is greater than a first threshold value,

activating, while being positioned at the first datum point, the surgical tool to perform the tissue modification task along the planned trajectory,

determining, based on force measurements on the surgical tool or a position of the tool along the planned trajectory, whether the surgical tool has reached an end of the planned trajectory,

in response to determining that the surgical tool has reached the end of the planned trajectory, retracting the surgical tool from the tissue.