AUTOMATIC IDENTIFICATION OF CONTENT SHARED DURING VIDEOCONFERENCES

1 . A method comprising:

accessing a video file that contains a plurality of frames of a recording of a videoconference,

wherein the recording of the videoconference contains a record of screen sharing by one or more participants in the videoconference;

generating, for each respective frame in an identified set of frames, a vector representation of an image of at least a portion of the respective frame;

wherein the identified set of frames includes one or more frames of the recording of the videoconference that have been classified as content-containing frames;

matching an image of a respective frame in the identified set of frames to a selected content item from a content repository, based on a degree of similarity between (i) the vector representation of the image of the respective frame in the identified set of frames, and (ii) a vector representation of the selected content item; and

storing a representation linking the video file to the selected content item.

2 . The method of claim 1 , further comprising:

applying a classification model to at least a subset of the plurality of frames,

wherein the classification model is configured to input an image of each frame in the subset of frames and to output, for each frame in the subset of frames, one of a set of predefined classifications of the frame, and

wherein the set of predefined classifications of the frame includes a content-containing-frame classification.

3 . The method of claim 2 , further comprising:

selecting the subset of the plurality of frames by:

generating a subsample of the plurality of frames;

for a consecutively subsampled first frame and second frame of the plurality of frames, applying the classification model to each of the first frame and the second frame to generate a first classification of the first frame and a second classification of the second frame;

if the first classification is a content-containing-frame classification, adding the first frame to the subset of the plurality of frames; and

if the second classification is a content-containing-frame classification, adding the second frame to the subset of the plurality of frames.

4 . The method of claim 3 , wherein selecting the subset of the plurality of frames further comprises:

in response to determining the first classification and second classification are different, performing a binary search of frames between the first frame and the second frame in the video file to identify a third frame at which a classification transitioned from the first classification to the second classification; and

adding the third frame to the subset of the plurality of frames.

5 . The method of claim 1 , further comprising:

classifying at least a subset of the plurality of frames based on metadata associated with the subset;

wherein the identified set of frames comprise one or more frames that are classified as content-containing frames based on the metadata of the one or more frames.

6 . The method of claim 1 , further comprising:

processing the identified set of frames to identify a transition from a first content item to a second content item.

7 . The method of claim 6 , wherein processing the identified set of frames comprises:

detecting, for a first frame and a second frame of the identified set of frames, that at least a threshold quantity of pixels are different between the first frame and the second frame;

wherein the first frame or the second frame is identified as the transition from the first content item to the second content item.

8 . The method of claim 1 , further comprising:

applying, to a first frame of the identified set of frames that is classified as content-containing frame, a bounding box model;

wherein the bounding box model defines, for the first frame, a region of the frame that contains a content item;

extracting an image of the first frame that includes the defined region of the frame;

wherein the image of the first frame is matched to the selected content item from the content repository.

9 . The method of claim 1 , further comprising:

identifying, for the respective frame, an identity of a user who is sharing content during the respective frame; and

accessing a set of candidate content items from the content repository with which the identified user interacted during a specified time period preceding a time of the videoconference;

wherein matching the image of the respective frame to the selected content item from the content repository comprises:

generating a measure of similarity between (i) the vector representation of the image of the respective frame and (ii) vector representations of each of the set of candidate content items.

10 . The method of claim 1 , further comprising:

storing, for a plurality of content items in the content repository, use data that describes user interactions with each of the plurality of content items; and

updating the use data associated with the selected content item in response to matching the image of the respective frame to the selected content item.

11 . The method of claim 1 , further comprising:

identifying a set of attendees of the videoconference; and

sending the selected content item to the identified attendees in response to matching the image of the respective frame to the selected content item.

12 . A content management system comprising:

one or more processors; and

one or more non-transitory computer readable media storing executable computer program instructions that, when executed by the one or more processors, cause the content management system to:

access a video file that contains a plurality of frames of a recording of a videoconference;

generate, for each respective frame in an identified set of frames, a vector representation of an image of at least a portion of the respective frame;

wherein the identified set of frames includes one or more frames of the recording of the videoconference that have been classified as content-containing frames;

match an image of a respective frame in the identified set of frames to a selected content item from a content repository, based on a degree of similarity between (i) the vector representation of the image of the respective frame in the identified set of frames, and (ii) a vector representation of the selected content item; and

store a representation linking the video file to the selected content item.

13 . The content management system of claim 12 , wherein execution of the instructions by the one or more processors further causes the content management system to:

apply a classification model to at least a subset of the plurality of frames,

wherein the classification model is configured to input an image of each frame in the subset of frames and to output, for each frame in the subset of frames, one of a set of predefined classifications of the frame, and

wherein the set of predefined classifications of the frame includes a content-containing-frame classification.

14 . The content management system of claim 12 , wherein execution of the instructions by the one or more processors further causes the content management system to:

process the identified set of frames to identify a transition from a first content item to a second content item, including:

detecting, for a first frame and a second frame of the identified set of frames, that at least a threshold quantity of pixels are different between the first frame and the second frame;

wherein the first frame or the second frame is identified as the transition from the first content item to the second content item.

15 . The content management system of claim 12 , wherein execution of the instructions by the one or more processors further causes the content management system to:

apply, to a first frame of the identified set of frames that is classified as content-containing frame, a bounding box model;

wherein the bounding box model defines, for the first frame, a region of the frame that contains a content item;

extract an image of the first frame that includes the defined region of the frame;

wherein the image of the first frame is matched to the selected content item from the content repository.

16 . The content management system of claim 12 , wherein execution of the instructions by the one or more processors further causes the content management system to:

identify a set of attendees of the videoconference; and

send the selected content item to the identified attendees in response to matching the image of the respective frame to the selected content item.

17 . A non-transitory computer readable medium storing executable computer program instructions that, when executed by one or more processors of a system, cause the system to:

access a video file that contains a plurality of frames of a recording of a videoconference;

generate, for each respective frame in an identified set of frames, a vector representation of an image of at least a portion of the respective frame;

wherein the identified set of frames includes one or more frames of the recording of the videoconference that have been classified as content-containing frames;

match an image of a respective frame in the identified set of frames to a selected content item from a content repository, based on a degree of similarity between (i) the vector representation of the image of the respective frame in the identified set of frames, and (ii) a vector representation of the selected content item; and

store a representation linking the video file to the selected content item.

18 . The non-transitory computer readable medium of claim 17 , wherein execution of the instructions by the one or more processors further causes the system to:

apply a classification model to at least a subset of the plurality of frames,

wherein the classification model is configured to input an image of each frame in the subset of frames and to output, for each frame in the subset of frames, one of a set of predefined classifications of the frame, and

wherein the set of predefined classifications of the frame includes a content-containing-frame classification.

19 . The non-transitory computer readable medium of claim 17 , wherein execution of the instructions by the one or more processors further causes the system to:

process the identified set of frames to identify a transition from a first content item to a second content item, including:

detecting, for a first frame and a second frame of the identified set of frames, that at least a threshold quantity of pixels are different between the first frame and the second frame;

wherein the first frame or the second frame is identified as the transition from the first content item to the second content item.

20 . The non-transitory computer readable medium of claim 17 , wherein execution of the instructions by the one or more processors further causes the system to:

apply, to a first frame of the identified set of frames that is classified as content-containing frame, a bounding box model;

wherein the bounding box model defines, for the first frame, a region of the frame that contains a content item;

extract an image of the first frame that includes the defined region of the frame;

wherein the image of the first frame is matched to the selected content item from the content repository.