Robotic surgical safety via video processing
One example method for improving robotic surgical safety via video processing includes identifying, during a robotic surgical procedure, one or more surgical tools using one or more images of the surgical procedure captured by a camera, the robotic surgical procedure employing a robotic surgical device controlling the one or more identified surgical tools; predicting, for at least one of the one or more images, one or more loaded surgical tools that are controlled by the robotic surgical device and should be in a field of view of the camera; comparing the one or more identified surgical tools with the one or more loaded surgical tools; determining that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools; and causing the at least one loaded tool of the robotic surgical device to be disabled.
1. A method comprising:
identifying, during a robotic surgical procedure, one or more surgical tools using one or more images of the robotic surgical procedure captured by a camera, the robotic surgical procedure employing a robotic surgical device controlling the one or more identified surgical tools;
predicting, for at least one of the one or more images, one or more loaded surgical tools that are controlled by the robotic surgical device and should be in a field of view of the camera;
comparing the one or more identified surgical tools with the one or more loaded surgical tools;
determining that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools; and
causing the at least one loaded tool of the robotic surgical device to be disabled.
2. The method of claim 1 , wherein identifying the one or more surgical tools using the one or more images comprises:
estimating a position of each of the one or more identified surgical tools; and
recognizing a type of each of the one or more surgical tools using the one or more images.
3. The method of claim 2 , wherein comparing the one or more identified surgical tools with the one or more loaded surgical tools comprises determining a correspondence between the identified surgical tools with the loaded surgical tools based, at least in part, upon the estimated positions of the identified surgical tools and predicted positions of the loaded surgical tools.
4. The method of claim 3 , wherein the predicted positions of the loaded surgical tools are obtained based, at least in part, upon a kinematic chain model of the robotic surgical device.
5. The method of claim 3 , wherein determining that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools comprises determining that the at least one loaded surgical tool has no correspondence in the one or more identified surgical tools.
6. The method of claim 3 , wherein determining that at least one of the loaded surgical tools does not match any of the one or more identified surgical tools comprises determining that a type of the at least one loaded surgical tool is different from a type of a corresponding tool in the one or more identified surgical tools.
7. The method of claim 1 , wherein identifying the one or more surgical tools from the one or more images comprises:
identifying one or more identifiers visible from the one or more images; and
interpreting the one or more identifiers to identify the one or more surgical tools.
8. The method of claim 1 , further comprising:
determining that at least one of the identified surgical tools has no correspondence in the one or more loaded surgical tools; and
causing the at least one identified surgical tool to be enabled.
9. A robotic surgical device comprising:
a processor;
a camera; and
a non-transitory computer-readable medium having processor-executable instructions stored thereupon, which, when executed by the processor, cause the processor to:
identify, using one or more images of a surgical procedure captured by the camera during a robotic surgical procedure, one or more surgical tools used in the surgical procedure;
predict, for at least one of the one or more images, one or more loaded surgical tools that are loaded during the surgical procedure and should be in a field of view of the camera;
compare the one or more identified surgical tools with the one or more loaded surgical tools;
determine that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools; and
cause the at least one loaded surgical tool of the robotic surgical device to be disabled.
10. The robotic surgical device of claim 9 , wherein identifying the one or more surgical tools using the one or more images comprises estimating a position of each of the one or more identified surgical tools.
11. The robotic surgical device of claim 9 , wherein determining that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools comprises determining that the at least one loaded surgical tool has no correspondence in the one or more identified surgical tools.
12. The robotic surgical device of claim 9 , wherein identifying the one or more surgical tools using the one or more images comprises recognizing a type of each of the one or more surgical tools using the one or more images.
13. The robotic surgical device of claim 9 , wherein determining that at least one of the loaded surgical tools does not match any of the one or more identified surgical tools comprises determining that a type of the at least one loaded surgical tool is different from a type of a corresponding tool in the one or more identified surgical tools.
14. A non-transitory computer-readable medium comprising processor-executable instructions to cause a processor to:
identify, during a robotic surgical procedure, one or more surgical tools using one or more images of the robotic surgical procedure captured by a camera, the robotic surgical procedure employing a robotic surgical device controlling the one or more identified surgical tools;
predict, for at least one of the one or more images, one or more loaded surgical tools that are controlled by the robotic surgical device and should be in a field of view of the camera;
compare the one or more identified surgical tools with the one or more loaded surgical tools;
determine that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools; and
cause the at least one loaded surgical tool of the robotic surgical device to be disabled.
15. The non-transitory computer-readable medium of claim 14 , wherein identifying the one or more surgical tools using the one or more images comprises estimating a position of each of the one or more identified surgical tools.
16. The non-transitory computer-readable medium of claim 14 , wherein determining that at least one of the one or more loaded surgical tools does not match any of the one or more identified surgical tools comprises determining that the at least one loaded surgical tool has no correspondence in the one or more identified surgical tools.
17. The non-transitory computer-readable medium of claim 14 , wherein determining that at least one of the loaded surgical tools does not match any of the one or more identified surgical tools comprises determining that a type of the at least one loaded surgical tool is different from a type of a corresponding tool in the one or more identified surgical tools.