Method to extract video context from external description

1 . A method, comprising:

for receipt of a video for processing:

executing a zero shot image labeler on the video to generate a plurality of labels corresponding to images of the video;

calculating an image embedding vector for each of the labels of the images;

generating context labels to replace the each of the labels corresponding to each of the images based on the context labels determined for current and previous images in time by referencing a context database with the image embedding vector to determine the context labels from an inspected event graph, wherein nodes of the event graph are indicative of events that can happen during a duration of the video; and

replacing the each of the plurality of labels with the generated context labels;

identifying a task from the generated context labels; and

determining a cycle time for the identified task from a length of the video across the generated context labels associated from the identified task.

2 . The method of claim 1 , wherein the context database manages an association between event embedding vectors and context information with each of the nodes in the inspected event graph.

3 . The method of claim 1 , wherein the generating context labels to replace the each of the labels corresponding to each of the images comprises determining a node from the inspected event graph associated with an event embedding vector in the context database having a smallest distance from the image embedding vector; and

generating the context labels from context information associated with the node.

4 . The method of claim 3 , wherein each node is associated with an estimated duration in the context database;

wherein a calculated distance between the image embedding vector and the event embedding vector is weighted based on a duration of time between the images of the video and the current and previous images in time associated with a same context label, in comparison to the estimated duration.

5 . The method of claim 1 , wherein the context database manages forbidden procedures in the inspected event graph, wherein the method further comprises, for the generated context labels including one of the forbidden procedures, raising an alert regarding non-compliance of standard operating procedure.

6 . The method of claim 1 , further comprising, for one or more of the generated context labels indicative of an abnormal event, providing an indication that the abnormal event has occurred.

7 . The method of claim 1 , further comprising:

storing the generated context labels with the video to index the video in a search engine.

8 . A system, comprising:

a context database; and

a processor, configured to:

for receipt of a video for processing:

execute an image labeler on the video to generate a plurality of labels corresponding to images of the video;

calculate a zero shot image embedding vector for each of the labels of the images;

generate context labels to replace the each of the labels corresponding to each of the images based on the context labels determined for current and previous images in time by referencing the context database with the image embedding vector to determine the context labels from an inspected event graph, wherein nodes of the event graph are indicative of events that can happen during a duration of the video;

replace the each of the plurality of labels with the generated context labels;

identify a task from the generated context labels; and

determine a cycle time for the identified task from a length of video across the generated context labels associated from the identified task.

9 . The system of claim 8 , wherein the context database manages an association between event embedding vectors and context information with each of the nodes in the inspected event graph.

10 . The system of claim 8 , wherein the processor is configured to generate context labels to replace the each of the labels corresponding to each of the images by determining a node from the inspected event graph associated with an event embedding vector in the context database having a smallest distance from the image embedding vector; and

generating the context labels from context information associated with the node.

11 . The system of claim 10 , wherein each node is associated with an estimated duration in the context database, wherein a calculated distance between the image embedding vector and the event embedding vector is weighted based on a duration of time between the images of the video and the current and previous images in time associated with a same context label, in comparison to the estimated duration.

12 . The system of claim 8 , wherein the context database manages forbidden procedures in the inspected event graph, wherein the instructions further comprise, for the generated context labels including one of the forbidden procedures, raising an alert regarding non-compliance of standard operating procedure.

13 . The system of claim 8 , wherein the processor is further configured to:

for one or more of the generated context labels indicative of an abnormal event, provide an indication that the abnormal event has occurred.

14 . The system of claim 8 , wherein the processor is further configured to:

storing the generated context labels with the video to index the video in a search engine.