Video frame type classification for a communication session
Methods and systems provide for providing video frame type classification in a communication session. In one embodiment, the system receives video content of a communication session with a number of participants; extracts frames from the video content; classifies the frames of the video content based on image analysis; and transmits, to one or more client devices, the classification of the frames of the video content.
1. A method, comprising:
receiving video content of a communication session comprising a plurality of participants;
extracting high-resolution versions and low-resolution versions of frames from the video content;
classifying the low-resolution versions of the frames of the video content into black frames, slide frames, and demo frames based on image analysis;
identifying text in the low-resolution versions of the frames;
extracting textual content from the high-resolution versions of the frames; and
transmitting, to one or more client devices, the extracted textual content.
2. The method of claim 1 , wherein the black frames are devoid of content, the slide frames include presentation slides, a face frame that does not include a screen share, and the demo frames include a demonstration.
3. The method of claim 2 , wherein textual content of the face frame is used for one or more of: a sentiment analysis presentation, and an engagement analysis presentation.
4. The method of claim 2 , wherein textual content of the demo frames are used for presentation of an analysis of a product demonstration duration within the communication session.
5. The method of claim 2 , wherein textual content of the demo frame is used for training data for one or more natural language parsing (NLP) tasks.
6. The method of claim 1 , wherein classifying the frames of the video content is performed using a convolutional neural network (CNN).
7. The method of claim 1 , further comprising:
post-processing the frames based on a classification of the frames.
8. The method of claim 7 , wherein the post-processing comprises:
determining a time between two neighboring frames that does not meet a length threshold; and
removing noise between the two neighboring frames.
9. The method of claim 1 , further comprising:
determining one or more differences in a classification in neighboring frames.
10. The method of claim 9 , further comprising:
segmenting the communication session into topic segments based on the determined differences in the classification in neighboring frames.
11. The method of claim 9 , further comprising:
presenting, to the one or more client devices, a visual indication of the differences in the classification in neighboring frames throughout the video content of the communication session.
12. A communication system comprising one or more processors configured to perform operations comprising:
receiving video content of a communication session comprising a plurality of participants;
extracting high-resolution versions and low-resolution versions of frames from the video content;
classifying the low-resolution versions of the frames of the video content into black frames that are devoid of content, slide frames, and demo frames based on image analysis;
identifying text in the low-resolution versions of the frames;
extracting textual content from the high-resolution versions of the frames; and
transmitting, to one or more client devices, the extracted textual content.
13. The communication system of claim 12 , wherein the slide frames include presentation slides and the demo frames include a demonstration.
14. The communication system of claim 12 , wherein the frames include a face frame that does not include a screen share, wherein textual content of the face frame is used for one or more of: a sentiment analysis presentation, and an engagement analysis presentation.
15. The communication system of claim 12 , wherein textual content of the demo frames are used for presentation of an analysis of a product demonstration duration within the communication session.
16. The communication system of claim 12 , wherein textual content of the demo frames are used for training data for one or more natural language parsing (NLP) tasks.
17. The communication system of claim 12 , wherein classifying the frames of the video content is performed using a convolutional neural network (CNN).
18. The communication system of claim 12 , further comprising:
post-processing the frames based on a classification of the frames.
19. The communication system of claim 18 , wherein the post-processing comprises:
determining a time between two neighboring frames that does not meet a length threshold; and
removing noise between the two neighboring frames.
20. A non-transitory computer-readable medium containing instructions that when executed by a processor, cause the processor to perform operations comprising:
receiving video content of a communication session comprising a plurality of participants;
extracting high-resolution versions and low-resolution versions of frames from the video content;
classifying the low-resolution versions of the frames of the video content into black frames that are devoid of content, slide frames, and demo frames based on image analysis;
identifying text in the low-resolution versions of the frames;
extracting textual content from the high-resolution versions of the frames; and
transmitting, to one or more client devices, the extracted textual content.