Artificial intelligence guidance system for robotic surgery
This invention is a system and method for utilizing artificial intelligence to operate a surgical robot (e.g., to perform a laminectomy), including a surgical robot, an artificial intelligence guidance system, an image recognition system, an image recognition database, and a database of past procedures with sensor data, electronic medical records, and imaging data. The image recognition system may identify the tissue type present in the patient and if it is the desired tissue type, the AI guidance system may remove a layer of that tissue with the end effector on the surgical robot, and have the surgeon define the tissue type if the image recognition system identified the tissue as anything other than the desired tissue type.
1. A method implemented in a computing system for at least partially controlling a portion of a robotic surgical apparatus, the method comprising:
obtaining a first image of a region of interest associated with a subject;
determining a first tissue type identification associated with at least a portion of the first image based, at least in part, upon a first output generated by applying at least a portion of the first image to a classifier model trained on a training set, the training set comprising data associated with depictions of tissue;
obtaining a second image of the region of interest after obtaining the first image;
determining a second tissue type identification associated with at least a portion of the second image based, at least in part, upon a second output generated by applying at least a portion of the second image to the classifier model; and
determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image.
2. The method of claim 1 , the method further comprising stopping at least a portion of a surgical operation based upon the determination that the fail-safe trigger has been fulfilled.
3. The method of claim 1 , wherein the first tissue type identification is one of a sequence of tissue type identifications determined prior to the obtaining of the second image, and wherein
determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises determining that no tissue type in the sequence of tissue type identifications corresponds to the tissue type of the second tissue type identification.
4. The method of claim 1 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises:
determining that the tissue type of the first tissue type identification is not the same tissue type as of the second tissue type identification; and
in response, at least in part, to determining that the tissue type of the first tissue type identification is not the same tissue type as of the second tissue type identification, determining that a time of acquisition of the second image is before a temporal threshold.
5. The method of claim 1 , wherein the first tissue type identification is one of a sequence of tissue type identifications determined prior to the obtaining of the second image, and wherein
determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises:
determining a temporal threshold based upon the sequence of tissue type identifications; and
determining that a time of acquisition of the second image is before the temporal threshold.
6. The method of claim 1 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises:
determining that a tissue type of the first tissue type identification is the same tissue type as of the second tissue type identification;
determining a first time of acquisition associated with the first image;
determining a second time of acquisition associated with the second image; and
determining that a difference between the first time of acquisition and the second time of acquisition exceeds a threshold.
7. The method of claim 1 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises determining that a first probability of classification associated with the first tissue type identification is greater than a second probability of classification associated with the second tissue type identification.
8. The method of claim 1 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises determining that the quality of the second image is degraded relative to the quality of the first image.
9. A non-transitory computer-readable storage medium comprising instructions configured, when executed by one or more processors, to cause a computer system to perform the following actions:
obtaining a first image of a region of interest associated with a subject;
determining a first tissue type identification associated with at least a portion of the first image based, at least in part, upon a first output generated by applying at least a portion of the first image to a classifier model trained on a training set, the training set comprising data associated with depictions of tissue;
obtaining a second image of the region of interest after obtaining the first image;
determining a second tissue type identification associated with at least a portion of the second image based, at least in part, upon a second output generated by applying at least a portion of the second image to the classifier model; and
determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image.
10. The non-transitory computer-readable storage medium of claim 9 , wherein the instructions are further configured, when executed by the one or more processors, to cause the computer system to stop at least a portion of a surgical operation based upon the determination that the fail-safe trigger has been fulfilled.
11. The non-transitory computer-readable storage medium of claim 9 , wherein the first tissue type identification is one of a sequence of tissue type identifications determined prior to the obtaining of the second image, and wherein
determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises determining that no tissue type in the sequence of tissue type identifications corresponds to the tissue type of the second tissue type identification.
12. The non-transitory computer-readable storage medium of claim 9 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises:
determining that the tissue type of the first tissue type identification is not the same tissue type as of the second tissue type identification; and
in response, at least in part, to determining that the tissue type of the first tissue type identification is not the same tissue type as of the second tissue type identification, determining that a time of acquisition of the second image is before a temporal threshold.
13. The non-transitory computer-readable storage medium of claim 9 , wherein the first tissue type identification is one of a sequence of tissue type identifications determined prior to the obtaining of the second image, and wherein
determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises:
determining a temporal threshold based upon the sequence of tissue type identifications; and
determining that a time of acquisition of the second image is before the temporal threshold.
14. The non-transitory computer-readable storage medium of claim 9 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises:
determining that a tissue type of the first tissue type identification is the same tissue type as of the second tissue type identification;
determining a first time of acquisition associated with the first image;
determining a second time of acquisition associated with the second image; and
determining that a difference between the first time of acquisition and the second time of acquisition exceeds a threshold.
15. The non-transitory computer-readable storage medium of claim 9 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises determining that a first probability of classification associated with the first tissue type identification is greater than a second probability of classification associated with the second tissue type identification.
16. The non-transitory computer-readable storage medium of claim 15 , wherein the classifier model is a probabilistic classifier.
17. The non-transitory computer-readable storage medium of claim 9 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises determining that the quality of the second image is degraded relative to the quality of the first image.
18. A system for at least partially controlling a robotic surgical apparatus, the system comprising:
at least one processor; and
a memory comprising instructions configured, when executed by the at least one processor, to cause the system to perform the following actions:
obtaining a first image of a region of interest associated with a subject;
determining a first tissue type identification associated with at least a portion of the first image based, at least in part, upon a first output generated by applying at least a portion of the first image to a classifier model trained on a training set, the training set comprising data associated with depictions of tissue;
obtaining a second image of the region of interest after obtaining the first image;
determining a second tissue type identification associated with at least a portion of the second image based, at least in part, upon a second output generated by applying at least a portion of the second image to the classifier model; and
determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image.
19. The system of claim 18 , wherein the instructions are further configured, when executed by the at least one processor, to cause the system to stop at least a portion of a surgical operation based upon the determination that the fail-safe trigger has been fulfilled.
20. The system of claim 18 , wherein the first tissue type identification is one of a sequence of tissue type identifications determined prior to the obtaining of the second image, and wherein
determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises determining that no tissue type in the sequence of tissue type identifications corresponds to the tissue type of the second tissue type identification.
21. The system of claim 18 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises:
determining that the tissue type of the first tissue type identification is not the same tissue type as of the second tissue type identification; and
in response, at least in part, to determining that the tissue type of the first tissue type identification is not the same tissue type as of the second tissue type identification, determining that a time of acquisition of the second image is before a temporal threshold.
22. The system of claim 18 , wherein the first tissue type identification is one of a sequence of tissue type identifications determined prior to the obtaining of the second image, and wherein
determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises:
determining a temporal threshold based upon the sequence of tissue type identifications; and
determining that a time of acquisition of the second image is before the temporal threshold.
23. The system of claim 18 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises:
determining that a tissue type of the first tissue type identification is the same tissue type as of the second tissue type identification;
determining a first time of acquisition associated with the first image;
determining a second time of acquisition associated with the second image; and
determining that a difference between the first time of acquisition and the second time of acquisition exceeds a threshold.
24. The system of claim 18 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises determining that a first probability of classification associated with the first tissue type identification is greater than a second probability of classification associated with the second tissue type identification.
25. The system of claim 18 , wherein determining that a fail-safe trigger has been fulfilled based, at least in part, upon the second image comprises determining that the quality of the second image is degraded relative to the quality of the first image.