Method and system for anonymizing raw surgical procedure videos

1. A computer-implemented method for anonymizing raw surgical procedure videos recorded by a recording device within an operating room (OR), the method comprising:

receiving a set of raw surgical videos corresponding to a surgical procedure performed within the OR;

merging the set of raw surgical videos to generate a surgical procedure video corresponding to the surgical procedure, wherein the surgical procedure video comprises a set of raw video images;

detecting image-based personally-identifiable information embedded in the set of raw video images, which includes detecting a type of image-based personally-identifiable information associated with one or more surgical tools used during the surgical procedure, wherein the set of raw video images includes images of the one or more surgical tools captured inside a patient's body when used to perform the surgical procedure; and

when image-based personally-identifiable information is detected, de-identifying the detected image-based personally-identifiable information in the surgical procedure video.

2. The computer-implemented method of claim 1 , wherein merging the set of raw surgical videos to generate the surgical procedure video includes:

analyzing a set of file names associated with the set of raw surgical videos to determine a correct order with respect to the surgical procedure; and

stitching together the set of raw surgical videos based on the determined order.

3. The computer-implemented method of claim 1 , wherein detecting the type of image-based personally-identifiable information associated with the one or more surgical tools further includes:

detecting personally-identifiable text printed on the one or more surgical tools in the images of the one or more surgical tools;

wherein if personally-identifiable text is detected in the images of the one or more surgical tools, de-identifying the detected image-based personally-identifiable information by blurring out or otherwise making the detected text unreadable in the images of the one or more surgical tools.

4. The computer-implemented method of claim 1 , wherein the image-based personally-identifiable information further includes:

text displayed inside the OR where the surgical procedure was performed, wherein the text was recorded by the recording device during an out-of-body (OOB) event when the recording device was taken out of the patient's body.

5. The computer-implemented method of claim 3 , wherein detecting the personally-identifiable text printed on the one or more surgical tools includes:

detecting, using a machine-learning-based tool-detection-and-recognition model, a surgical tool within a first subset of the set of raw video images; and

processing a portion of the first subset of the set of raw video images containing the detected surgical tool to detect any personally-identifiable text within the portion of the first subset of the set of raw video images.

6. The computer-implemented method of claim 1 , wherein the image-based personally-identifiable information further includes a text box inserted into the surgical procedure video to display surgical procedure-related information.

7. The computer-implemented method of claim 6 , wherein detecting the text-box in the surgical procedure video includes:

detecting, using a machine-learning-based text-box detection model, a text box at or in the vicinity of a predetermined location within a second subset of the set of raw video images; and

processing a portion of the second subset of the set of raw video images containing the detected text box to detect any personally-identifiable text within the portion of the second subset of the set of raw video images.

8. The computer-implemented method of claim 1 , wherein detecting the image-based personally-identifiable information embedded in the set of raw video images of the surgical procedure video further includes:

scanning the set of raw video images to detect a video segment corresponding to an OOB event when the recording device was taken out of a patient's body, wherein personally-identifiable information can be captured by the recording device during the OOB event; and

wherein if an OOB event is detected within the set of raw video images, de-identifying the detected image-based personally-identifiable information by automatically blurring out or otherwise edited out each video image in the detected video segment corresponding to the detected OOB event, so that any personally-identifiable information embedded within the detected video segment cannot be identified.

9. The computer-implemented method of claim 8 , wherein detecting a video segment corresponding to an OOB event in the set of raw video images includes:

detecting, using a machine-learning-based OOB-event detection model, a beginning phase of an OOB event when an endoscope camera is being taken out of a patient's body;

detecting, using the machine-learning-based OOB-event detection model, an ending phase of the OOB event when the endoscope camera is being inserted back into the patient's body; and

marking a set of video images between the detected beginning phase and the detected ending phase in the set of raw video images as the video segment corresponding to the detected OOB event.

10. The computer-implemented method of claim 9 , wherein prior to using the machine-learning-based OOB-event detection model to detect an OOB event, the method further comprises training the OOB-event detection model based on a set of labeled video segments of a set of OOB events extracted from actual surgical procedure videos.

11. The computer-implemented method of claim 1 , wherein prior to merging the set of raw surgical videos, the method further includes:

processing the received set of raw surgical videos to detect text-based personally-identifiable information embedded in file-structure data associated with the set of raw surgical videos; and

if text-based personally-identifiable information is detected, removing or otherwise de-identifying the detected text-based personally-identifiable information from the file-structure data.

12. The computer-implemented method of claim 11 , wherein the file-structure data include:

file identifiers associated with the set of raw surgical videos;

folder identifiers of a file folder containing the set of raw surgical videos;

file properties associated with the set of raw surgical videos; and

other metadata associated with the set of raw surgical videos.

13. The computer-implemented method of claim 1 , wherein after de-identifying the detected image-based personally-identifiable information in the surgical procedure video, the method further includes:

performing a random sampling within the de-identified surgical procedure video to randomly select a number of video segments within the de-identified surgical procedure video; and

verifying that the randomly selected video segments are free of any personally-identifiable information.

14. The computer-implemented method of claim 1 , wherein after de-identifying the detected image-based personally-identifiable information in the surgical procedure video, the method further includes:

permanently removing the set of raw surgical videos from a surgical video repository; and

replacing the removed raw surgical videos with de-identified surgical procedure video.

15. The computer-implemented method of claim 1 , wherein the personally-identifiable information includes both patient identifiable information associate with a patient receiving the surgical procedure and surgical staff identifiable information associated with surgical staff performing the surgical procedure.

16. A system for anonymizing raw surgical procedure videos recorded by a recording device within an operating room (OR), the system comprising:

one or more processors;

a memory coupled to the one or more processors; and

wherein the memory stores a set of instructions that, when executed by the one or more processors, cause the system to:

receive a set of raw surgical videos corresponding to a surgical procedure performed within the OR;

merge the set of raw surgical videos to generate a surgical procedure video corresponding to the surgical procedure, wherein the surgical procedure video comprises a set of raw video images;

detect personally-identifiable information embedded in the set of raw video images of the surgical procedure video, which includes detecting a type of image-based personally-identifiable information associated with one or more surgical tools used during the surgical procedure, wherein the set of raw video images includes images of the one or more surgical tools captured inside a patient's body when used to perform the surgical procedure; and

when personally-identifiable information is detected, de-identify the detected personally-identifiable information in the surgical procedure video.

17. The system of claim 16 , wherein detecting the type of personally-identifiable information associated with the one or more surgical tools further includes:

detecting personally-identifiable text printed on the one or more surgical tools in the images of the one or more surgical tools; and

wherein if personally-identifiable text is detected in the images of the one or more surgical tools, de-identifying the detected personally-identifiable information includes blurring out or otherwise making the detected text unreadable in the images of the one or more surgical tools.

18. The system of claim 16 ,

wherein detecting the personally-identifiable information embedded in the set of raw video images of the surgical procedure video further includes scanning the set of raw video images to detect a video segment corresponding to an OOB event when the recording device was taken out of a patient's body where the surgical procedure was performed; and

wherein if an OOB event is detected within the set of raw video images, de-identifying the detected personally-identifiable information by automatically blurring out or otherwise edited out each video image in the detected video segment corresponding to the detected OOB event, so that any personally-identifiable information embedded within the detected video segment cannot be identified.

19. The system of claim 16 , wherein after de-identifying the detected personally-identifiable information in the surgical procedure video, the system is configured to:

permanently remove the set of raw surgical videos from a surgical video repository; and

replace the removed raw surgical videos with de-identified surgical procedure video.