Systems and methods for segmenting surgical videos

1. A method comprising:

receiving, by a processor of a computing device, surgical video, the surgical video comprising at least a sequence of video frames captured by an endoscope during an endoscopic surgical procedure;

recognizing one or more steps of the surgical video by the processor using a trained ML technique;

identifying one or more video frames of the surgical video corresponding to the recognized steps;

in response to identification of the one or more video frames, generating one or more bookmarks based on the one or more steps of the surgical video and the one or more video frames;

associating, by the processor, the one or more bookmarks with the respective one or more video frames; and

storing, by the processor, the one or more bookmarks.

2. The method of claim 1 , further comprising:

determining, by the processor, an event in the surgical video; and

determining the identification of the video frame based on the event.

3. The method of claim 2 , wherein the event comprises one of changing a surgical robotic tool or activating a surgical robotic tool.

4. The method of claim 1 , further comprising:

receiving, by the processor, audio information from a microphone; and

determining the identification of the video frame based on the audio information.

5. The method of claim 4 , further comprising recognizing, by the processor using a speech recognition technique, one or more spoken words from the audio information; and

determining the identification of the video frame based on the one or more spoken words.

6. The method of claim 4 , wherein the audio information is received during the endoscopic surgical procedure and wherein determining the identification of the video frame happens substantially in real-time during the endoscopic surgical procedure.

7. The method of claim 1 , further comprising:

receiving, by the processor, input information from an input device; and

determining the identification of the video frame based on the input information.

8. The method of claim 7 , further comprising:

determining, by the processor, a gesture based on the input information; and

wherein determining the identification of the video frame is based on the gesture.

9. The method of claim 1 , further comprising:

receiving, by the processor, input information from an input device;

generating an additional bookmark based on the input information; and

storing the additional bookmark.

10. A method comprising:

receiving, by a processor of a computing device, a surgical video having a bookmark;

receiving a selection of the bookmark;

receiving an indication to share a segment of the surgical video and a recipient, the segment of the surgical video based on the bookmark;

extracting a portion of the surgical video based on the bookmark;

storing the extracted portion of the surgical video; and

providing the stored extracted portion of the surgical video to the recipient.

11. The method of claim 10 , wherein the surgical video has a plurality of bookmarks and the bookmark is a first bookmark of the plurality of bookmarks, and further comprising:

receiving a selection of a second bookmark of the plurality of bookmarks; and

wherein extracting the portion of the surgical video is further based on the second bookmark.

12. The method of claim 11 , further comprising:

receiving, by the processor, input information from an input device; and

generating the second bookmark based on the input information.

13. The method of claim 10 , wherein the bookmark corresponds to a frame in the surgical video, and further comprising receiving a scrubbing input to change the bookmark to a different frame in the surgical video.

14. The method of claim 10 , wherein the recipient comprises a user, and wherein providing the extracted portion of the surgical video comprises one or more of emailing a link to the portion of the surgical video to the recipient or emailing a copy of the extracted portion of the surgical video to the recipient.

15. The method of claim 10 , wherein the recipient comprises a web portal and providing the extracted portion of the surgical video to the recipient comprises posting the extracted portion of the surgical video to the web portal.

16. A system comprising:

a non-transitory computer-readable medium; and

one or more processor communicatively coupled to the non-transitory computer-readable medium, the processor configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:

receive surgical video, the surgical video comprising at least a sequence of video frames captured by an endoscope during an endoscopic surgical procedure;

recognize one or more steps of the surgical video by the processor using a trained ML technique;

identify one or more video frames of the surgical video corresponding to the recognized steps;

in response to identification of the one or more video frames, generate one or more bookmarks based on the one or more steps of the surgical video and the one or more video frames;

associate the one or more bookmarks with the respective one or more video frames; and

store the one or more bookmarks.

17. The system of claim 16 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

determine an event in the surgical video; and

determine the identification of the video frame based on the event.

18. The system of claim 17 , wherein the event comprises one of changing a surgical robotic tool or activating a surgical robotic tool.

19. The system of claim 16 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

receive audio information from a microphone; and

determine the identification of the video frame based on the audio information.

20. The system of claim 19 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

recognize, using a speech recognition technique, one or more spoken words from the audio information; and

determine the identification of the video frame based on the one or more spoken words.

21. The system of claim 20 , wherein the audio information is received during the endoscopic surgical procedure and wherein determination of the identification of the video frame happens substantially in real-time during the endoscopic surgical procedure.

22. The system of claim 16 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

receive input information from an input device; and

determine the identification of the video frame based on the input information.

23. The system of claim 22 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

determine a gesture based on the input information; and

determine the identification of the video frame based on the gesture.

24. The system of claim 16 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

receive input information from an input device;

generate an additional bookmark based on the input information; and

store the additional bookmark.

25. A system comprising:

a non-transitory computer-readable medium; and

one or more processor communicatively coupled to the non-transitory computer-readable medium, the processor configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:

receive a surgical video having a bookmark;

receive a selection of the bookmark;

receive an indication to share a segment of the surgical video and a recipient, the segment of the surgical video based on the bookmark;

extract a portion of the surgical video based on the bookmark;

store the extracted portion of the surgical video; and

provide the stored extracted portion of the surgical video to the recipient.

26. The system of claim 25 , wherein the surgical video has a plurality of bookmarks and the bookmark is a first bookmark of the plurality of bookmarks, and wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

receive a selection of a second bookmark of the plurality of bookmarks; and

extract the portion of the surgical video further based on the second bookmark.

27. The system of claim 26 , wherein the surgical video has a plurality of bookmarks and the bookmark is a first bookmark of the plurality of bookmarks, and wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

receive input information from an input device; and

generate the second bookmark based on the input information.

28. The system of claim 25 , wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to receive a scrubbing input to scrub the bookmark to a frame in the surgical video.

29. The system of claim 25 , wherein the recipient comprises a user, and wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to:

email a link to the extracted portion of the surgical video to the recipient; or

email a copy of the extracted portion of the surgical video to the recipient.

30. The system of claim 25 , wherein the recipient comprises a web portal and wherein the processor is configured to execute further processor-executable instructions stored in the non-transitory computer-readable medium to post the extracted portion of the surgical video to the web portal.