Collaborative video application for remote servicing
View Patent ↗One embodiment of the present invention provides a system for sharing annotated videos. During operation, the system establishes a real-time video-sharing session between a remote field computer and a local computer. During the established real-time video-sharing session, the system receives a real-time video stream from a remote field computer, forwards the real-time video stream to a local computer to allow an expert to provide an annotation to the real-time video stream, receives the annotation from the local computer, and forwards the annotation to the remote field computer, which associates the annotation with a corresponding portion of the real-time video stream and displays the annotation on top of the corresponding portion of the real-time video stream.
1. A computer-executable method for sharing annotated videos, the method comprising:
establishing, by a collaboration server, a real-time video-sharing session between a remote field computer and a client computer associated with an expert on a procedure; and
during the established real-time video-sharing session:
receiving a real-time video stream from the remote field computer;
forwarding the real-time video stream to the client computer to allow the expert to provide an annotation to the real-time video stream;
receiving, from the client computer, a command to extract a still image from the video stream;
extracting the still image from the video stream in response to the received command;
forwarding the still image to the client computer for annotation by the expert;
receiving the annotation associated with the procedure from the client computer; and
forwarding the annotation to the remote field computer, which associates the annotation with a corresponding portion of the real-time video stream and displays the annotation on top of the corresponding portion of the real-time video stream.
2. The method of claim 1 , wherein forwarding the annotation involves:
generating a message associated with the annotation; and
sending the message to the remote field computer.
3. The method of claim 2 , wherein generating the message involves generating a JavaScript Object Notation (JSON) message that includes the annotation.
4. The method of claim 1 , wherein establishing the real-time video-sharing session involves creating a long-polling HTTP connection and/or establishing a WebSocket connection for the real-time video-sharing session.
5. The method of claim 1 , wherein the remote field computer includes at least one of:
a head-mounted computer;
a smartphone;
a tablet computer; and
a laptop computer.
6. The method of claim 1 , wherein the annotation is a HyperText Markup Language (HTML) canvas object.
7. The method of claim 1 , further comprising:
sending a set of extracted media segments associated with a step of the procedure, from multiple real-time video-sharing sessions, to a client computer of an expert, to allow the expert to generate a step-by-step procedure guide for the procedure; and
receiving the step-by-step procedure guide from the client computer, wherein the step-by-step procedure guide includes the set of media segments and a description associated with the step.
8. The method of claim 1 , further comprising facilitating an exchange of voice messages between the remote field computer and the local computer.
9. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for sharing annotated videos, the method comprising:
establishing a real-time video-sharing session between a remote field computer and a client computer associated with an expert on a procedure; and
during the established real-time video-sharing session:
receiving a real-time video stream from the remote field computer;
forwarding the real-time video stream to the client computer to allow the expert to provide an annotation to the real-time video stream;
receiving, from the client computer, a command to extract a still image from the video stream;
extracting the still image from the video stream in response to the received command;
forwarding the still image to the client computer for annotation by the expert;
receiving the annotation associated with the procedure from the client computer; and
forwarding the annotation to the remote field computer, which associates the annotation with a corresponding portion of the real-time video stream and displays the annotation on top of the corresponding portion of the real-time video stream.
10. The non-transitory computer-readable storage medium of claim 9 , wherein forwarding the annotation involves:
generating a message associated with the annotation; and
sending the message to the remote field computer.
11. The non-transitory computer-readable storage medium of claim 10 , wherein generating the message involves generating a JavaScript Object Notation (JSON) message that includes the annotation.
12. The non-transitory computer-readable storage medium of claim 9 , wherein establishing the real-time video-sharing session involves creating a long-polling HTTP connection and/or establishing a WebSocket connection for the real-time video-sharing session.
13. The non-transitory computer-readable storage medium of claim 9 , wherein the remote field computer includes at least one of:
a head-mounted computer;
a smartphone;
a tablet computer; and
a laptop computer.
14. The non-transitory computer-readable storage medium of claim 9 , wherein the annotation is a HyperText Markup Language (HTML) canvas object.
15. The non-transitory computer-readable storage medium of claim 9 , wherein the method further comprises:
sending a set of extracted media segments associated with a step of the procedure, from multiple real-time video-sharing sessions, to a client computer of an expert, to allow the expert to generate a step-by-step procedure guide for the procedure; and
receiving the step-by-step procedure guide from the client computer, wherein the step-by-step procedure guide includes the set of media segments and a description associated with the step.
16. The non-transitory computer-readable storage medium of claim 9 , wherein the method further comprises facilitating an exchange of voice messages between the remote field computer and the local computer.
17. A computing system for sharing annotated videos, the system comprising:
a remote field computer;
a client computer associated with an expert on a procedure; and
a video-sharing collaboration server, further comprising:
a session-establishing mechanism configured to establish a real-time video-sharing session between a remote field computer and a client computer associated with an expert on a procedure;
a video-receiving mechanism configured to receive, during the real-time video-sharing session, a real-time video stream from the remote field computer;
a video-forwarding mechanism configured to forward, during the real-time video-sharing session, the real-time video stream to the client computer to allow the expert to provide an annotation to the real-time video stream;
a command-receiving mechanism, configured to receive from the client computer, during the real-time video-sharing session, a command to extract a still image from the video stream;
an image-extracting mechanism, configured to extract, during the real-time video-sharing session, a still image from the video stream in response to the received command;
an image-forwarding mechanism, configured to forward, during the real-time video-sharing session, the still image to the client computer for annotation by the expert;
an annotation-receiving mechanism configured to receive, during the real-time video-sharing session, the annotation associated with the procedure from the client computer; and
an annotation-forwarding mechanism configured to forward the annotation to the remote field client, which associates the annotation with a corresponding portion of the real-time video stream.
18. The system of claim 17 , wherein while forwarding the annotation, the annotation-forwarding mechanism is further configured to:
generate a message associated with the annotation; and
send the message to the remote field computer.
19. The system of claim 18 , wherein generating the message involves generating a JavaScript Object Notation (JSON) message that includes the annotation.
20. The system of claim 17 , wherein the session-establishing mechanism is configured to create a long-polling HTTP connection and/or establish a WebSocket connection for the real-time video-sharing session.
21. The system of claim 17 , wherein the remote field computer includes at least one of:
a head-mounted computer;
a smartphone;
a tablet computer; and
a laptop computer.
22. The system of claim 17 , wherein the annotation is a HyperText Markup Language (HTML) canvas object.
23. The system of claim 17 , wherein the video-sharing server further comprises:
a media-sending mechanism configured to send a set of extracted media segments associated with a step of the procedure, from multiple real-time video-sharing sessions, to a client computer of an expert, to allow the expert to generate a step-by-step procedure guide for the procedure; and
a procedure-receiving mechanism configured to receive the step-by-step procedure guide from the client computer, wherein the step-by-step procedure guide includes the set of media segments and a description associated with the step.
24. The system of claim 17 , wherein the video-sharing server further comprises a mechanism that facilitates an exchange of voice messages between the remote field computer and the local computer.
25. The method of claim 17 , further comprising storing the annotation and at least an extracted media segment of the real-time video stream in an indexed media store in association with the procedure, wherein the captured media segment includes the still image or a captured video clip.