Method and system for automatically tracking and managing inventory of surgical tools in operating rooms
Embodiments described herein provide various examples of automatically processing surgical videos to detect surgical tools and tool-related events, and extract surgical-tool usage information. In one aspect, a process for automatically tracking usages of robotic surgery tools is disclosed. This process can begin by receiving a surgical video captured during a robotic surgery. The process then processes the surgical video to detect a surgical tool in the surgical video. Next, the process determines whether the detected surgical tool has been engaged in the robotic surgery. If so, the process further determines whether the detected surgical tool is engaged for a first time in the robotic surgery. If the detected surgical tool is engaged for the first time, the process subsequently increments a total-engagement count of the detected surgical tool. Otherwise, the process continues monitoring the detected surgical tool in the surgical video.
1 . A computer-implemented method for automatically tracking usages of surgical tools, the method comprising:
operating a robotic arm using a limited-use surgical tool;
receiving a surgical video captured during a surgery;
processing the surgical video to detect the limited-use surgical tool in the surgical video;
in response to detecting the limited-use surgical tool, activating a tool-usage counting process to track a counted number of uses of the limited-use surgical tool as used by the robotic arm during a tool engagement, wherein activating the tool-usage counting process to track the counted number of uses of the limited-use surgical tool includes activating a machine-learning model trained to detect an event in the surgical video corresponding to a single use of the limited-use surgical tool;
retrieving a stored number of total uses of the limited-use surgical tool accumulated from previous engagements of the limited-use surgical tool;
determining whether a combined number of uses of the counted number of uses during the tool engagement and the stored number of total uses has reached a predetermined number of uses determined based on a maximum number of safe uses of the limited-use surgical tool or a statistical number of effective uses of the limited-use surgical tool before the limited-use surgical tool becomes unstable; and
if so, generating a tool-expiration warning to surgical staff.
2 . The computer-implemented method of claim 1 , wherein the limited-use surgical tool is a surgical stapler, wherein a single use of the surgical stapler is a single firing of the surgical stapler, and wherein the machine-learning model is trained to detect a single firing of the surgical stapler by detecting an event when a firing knob of the surgical stapler is pushed from a first position to a second position.
3 . The computer-implemented method of claim 2 , further comprising using the counted number of uses of the surgical stapler during the tool engagement to infer a length of a tissue traversed by the surgical stapler.
4 . The computer-implemented method of claim 1 , wherein the limited-use surgical tool is an electrocautery tool, wherein a single use of the electrocautery tool is a single firing of the electrocautery tool, and wherein the machine-learning model is trained to detect a single firing of the electrocautery tool by detecting each occurrence of a plume of surgical smoke.
5 . The computer-implemented method of claim 1 , wherein the surgery is a robotic surgery, and wherein the limited-use surgical tool is a robotic surgery tool comprising a tool controller and a tool memory.
6 . The computer-implemented method of claim 5 ,
wherein the stored number of total uses of the limited-use surgical tool is retrieved from the tool memory by the tool controller, and
the combined number of uses of the limited-use surgical tool is written back into the tool memory by the tool controller to replace the stored number of total uses of the limited-use surgical tool.
7 . The computer-implemented method of claim 1 , further comprising:
in response to determining that the combined number of uses has not reached the predetermined number of uses, updating the stored number of total uses with the combined number of uses, and continuing monitoring the limited-use surgical tool during the surgery.
8 . The computer-implemented method of claim 1 , further comprising, in response to determining that the combined number of uses has reached the predetermined number of uses, marking the limited-use surgical tool as expired.
9 . The computer-implemented method of claim 1 , further comprising:
prior to activating a tool-usage counting process, determining whether the limited-use surgical tool has been engaged in the surgery by determining if the limited-use surgical tool continues to present in the surgical video for at least a predetermined minimum time period.
10 . The computer-implemented method of claim 1 , further comprising using the counted number of uses of the limited-use surgical tool during the tool engagement for post-operation analysis.
11 . The computer-implemented method of claim 1 , wherein processing the surgical video includes using computer-vision to detect the limited-use surgical tool in the surgical video.
12 . The computer-implemented method of claim 1 , wherein the tool-expiration warning provides an expiration alert communicated to surgical staff.
13 . An apparatus for automatically tracking usages of surgical tools, the apparatus comprising:
one or more processors; and
a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
operate a robotic arm using a limited-use surgical tool;
receive a surgical video captured during a surgery;
process the surgical video to detect the limited-use surgical tool in the surgical video;
in response to detecting the limited-use surgical tool, activate a tool-usage counting process to track a counted number of uses of the limited-use surgical tool as used by the robotic arm during a tool engagement;
retrieve a stored number of total uses of the limited-use surgical tool accumulated from previous engagements of the limited-use surgical tool; and
determine whether a combined number of uses of the counted number of uses during the tool engagement and the stored number of total uses has reached a predetermined number of uses determined based on a maximum number of safe uses of the limited-use surgical tool or a statistical number of effective uses of the limited-use surgical tool before the limited-use surgical tool becomes unstable, and if so then generate a tool-expiration warning to surgical staff,
wherein the surgery is a robotic surgery, the limited-use surgical tool is a robotic surgery tool comprising a tool controller and a tool memory, the stored number of total uses of the limited-use surgical tool is retrieved from the tool memory by the tool controller, and the combined number of uses of the limited-use surgical tool is written back into the tool memory by the tool controller to replace the stored number of total uses of the limited-use surgical tool.
14 . The apparatus of claim 13 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to activate the tool-usage counting process by activating a machine-learning model trained to detect an event in the surgical video corresponding to a single use of the limited-use surgical tool.
15 . The apparatus of claim 13 , wherein in response to determining that the combined number of uses has not reached the predetermined number of uses, the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
update the stored number of total uses with the combined number of uses; and
continue monitoring the limited-use surgical tool during the surgery.
16 . The apparatus of claim 13 , wherein the surgery is a robotic surgery and the limited-use surgical tool is a robotic surgery tool comprising a tool memory, and wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
retrieve the stored number of total uses of the limited-use surgical tool from the tool memory; and
write back the combined number of uses of the limited-use surgical tool into the tool memory to replace the stored number of total uses.
17 . An apparatus for automatically tracking usages of surgical tools, the apparatus comprising:
one or more processors; and
a memory coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
operate a robotic arm using a limited-use surgical tool;
receive a surgical video captured during a surgery;
process the surgical video to detect the limited-use surgical tool in the surgical video;
in response to detecting the limited-use surgical tool, activate a tool-usage counting process to track a counted number of uses of the limited-use surgical tool as used by the robotic arm during a tool engagement, wherein activating the tool-usage counting process includes activating a machine-learning model trained to detect an event in the surgical video corresponding to a single use of the limited-use surgical tool;
retrieve a stored number of total uses of the limited-use surgical tool accumulated from previous engagements of the limited-use surgical tool;
determine whether a combined number of uses of the counted number of uses during the tool engagement and the stored number of total uses has reached a predetermined number of uses determined based on a maximum number of safe uses of the limited-use surgical tool or a statistical number of effective uses of the limited-use surgical tool before the limited-use surgical tool becomes unstable; and
if so, generate a tool-expiration warning to surgical staff.
18 . The apparatus of claim 17 , wherein the limited-use surgical tool is a surgical stapler, wherein a single use of the surgical stapler is a single firing of the surgical stapler, and wherein the machine-learning model is trained to detect a single firing of the surgical stapler by detecting an event when a firing knob of the surgical stapler is pushed from a first position to a second position.
19 . The apparatus of claim 17 , wherein the limited-use surgical tool is an electrocautery tool, wherein a single use of the electrocautery tool is a single firing of the electrocautery tool, and wherein the machine-learning model is trained to detect a single firing of the electrocautery tool by detecting each occurrence of a plume of surgical smoke.
20 . A computer-implemented method for automatically tracking usages of surgical tools, the method comprising:
operating a robotic arm using a limited-use surgical tool;
receiving a surgical video captured during a surgery;
processing the surgical video to detect the limited-use surgical tool in the surgical video;
in response to detecting the limited-use surgical tool, activating a tool-usage counting process to track a counted number of uses of the limited-use surgical tool as used by the robotic arm during a tool engagement;
retrieving a stored number of total uses of the limited-use surgical tool accumulated from previous engagements of the limited-use surgical tool; and
determining whether a combined number of uses of the counted number of uses during the tool engagement and the stored number of total uses has reached a predetermined number of uses determined based on a maximum number of safe uses of the limited-use surgical tool or a statistical number of effective uses of the limited-use surgical tool before the limited-use surgical tool becomes unstable, and if so then generate a tool-expiration warning to surgical staff,
wherein the surgery is a robotic surgery, the limited-use surgical tool is a robotic surgery tool comprising a tool controller and a tool memory, the stored number of total uses of the limited-use surgical tool is retrieved from the tool memory by the tool controller, and the combined number of uses of the limited-use surgical tool is written back into the tool memory by the tool controller to replace the stored number of total uses of the limited-use surgical tool.
21 . The computer-implemented method of claim 20 , wherein activating the tool-usage counting process to track the counted number of uses of the limited-use surgical tool includes activating a machine-learning model trained to detect an event in the surgical video corresponding to a single use of the limited-use surgical tool.
22 . The computer-implemented method of claim 20 , further comprising:
in response to determining that the combined number of uses has not reached the predetermined number of uses, updating the stored number of total uses with the combined number of uses, and continuing monitoring the limited-use surgical tool during the surgery.