Automated recording highlights for conferences
A transcript of a conference (e.g., a video conference, an audio conference, or a telephone call with two or more participants) is processed to extract a conference summary. Scores are determined for strings of the transcript that are used to select strings for inclusion in the conference summary. Determining the scores includes determining respective sentence vectors for strings. A sentence vector has elements corresponding to words in the transcript that are proportional to occurrences of the word in the string and inversely proportional to occurrences of the word in the transcript. A short video conference summary or a short audio conference summary is then generated using timestamps from the transcript associated with strings (e.g., sentences) that have been selected for inclusion in the conference summary. The short video or audio summary may be presented to users to enable efficient storage and transmission of conference information within a unified communications system.
1 . A method comprising:
obtaining a transcript of a conference, wherein the transcript includes strings with respective timestamps;
removing stop words from the strings of the transcript;
selecting, from amongst the strings of the transcript, a first set of strings of the transcript with a number of remaining words, after the stop words are removed, greater than a threshold for determination of respective sentence vectors;
determining respective sentence vectors for the first set of strings of the transcript, wherein a sentence vector has elements corresponding to words present in the transcript that are proportional to a number of occurrences of a word in the string corresponding to the sentence vector and inversely proportional to a number of occurrences of the word in the transcript;
determining respective scores for the first set of strings of the transcript based on the respective sentence vectors;
determining respective scores for a second set of strings of the transcript based on a number of remaining words, after the stop words are removed, being less than the threshold;
selecting a selected string for highlighting from the transcript based on respective scores of strings;
selecting a selected video excerpt from a video of the conference based on the respective timestamp of the selected string, wherein the selected video excerpt includes multiple frames of video; and
generating a video conference summary as a sequence of video excerpts from the video, including the selected video excerpt.
2 . The method of claim 1 , wherein determining respective scores for the first set of strings of the transcript based on the respective sentence vectors comprises:
determining pairwise similarity scores of the respective sentence vectors, wherein one of the pairwise similarity scores is determined based one or more of a dot product, a Euclidean distance, a Pearson Correlation, a Jaccard coefficient, and a Tanimoto coefficient between two of the respective sentence vectors; and
determining the respective score for one of the strings based on a graph with vertices corresponding to the strings of the transcript and to edge weights corresponding to the pairwise similarity scores.
3 . The method of claim 1 , wherein a non-zero element of the respective sentence vector for one of the strings of the transcript is a term frequency-inverse document frequency for a word associated with the non-zero element.
4 . The method of claim 1 , wherein the strings of the transcript have respective speaker identifiers, and further comprising:
identifying speaker segments with respective durations in the transcript, wherein a speaker segment is a sequence of consecutive strings in the transcript that have a same speaker identifier;
selecting a speaker segment from the transcript based on a respective duration of the speaker segment; and
selecting the string for highlighting from the selected speaker segment based on respective scores of strings in the speaker segment.
5 . The method of claim 1 , wherein the strings of the transcript have respective speaker identifiers and the respective speaker identifier for the selected string is associated with a role identifier, and further comprising:
selecting the string for highlighting from the transcript based on the role identifier.
6 . The method of claim 1 , further comprising:
generating a highlighted transcript as a copy of the transcript with a subset of the strings highlighted, wherein the selected string is highlighted;
presenting the highlighted transcript to a user;
receiving user edits to the highlighting of the highlighted transcript; and
selecting the selected video excerpt from the video of the conference based on the respective timestamp of a string selected based on the user edits to the highlighting.
7 . The method of claim 1 , further comprising:
detecting one or more words from a set of keywords in a string from the transcript, wherein the selected string is selected based on presence of the one or more words from the set of keywords.
8 . The method of claim 1 , further comprising:
detecting an action item phrase in a string from the transcript, wherein the selected string is selected based on presence of the action item phrase.
9 . The method of claim 8 , wherein detecting the action item phrase in a string from the transcript comprises:
inputting the strings from the transcript to a machine learning classifier that has been trained to output predictions of whether a string includes an action item phrase.
10 . The method of claim 1 , wherein determining respective scores for the first set of strings of the transcript based on the respective sentence vectors comprises determining pairwise dot products of the respective sentence vectors, and determining the respective score for one of the strings in the first set of strings of the transcript based on a sum of the pairwise dot products for the respective sentence vector of the one of the strings in the first set of strings of the transcript.
11 . A system comprising:
a processor, and
a memory, wherein the memory stores instructions executable by the processor to:
obtain a transcript of a conference, wherein the transcript includes strings with respective timestamps;
remove stop words from the strings of the transcript;
select, from amongst the strings of the transcript, a first set of strings of the transcript with a number of remaining words, after the stop words are removed, greater than a threshold for determination of respective sentence vectors;
determine respective sentence vectors for the first set of strings of the transcript, wherein a sentence vector has elements corresponding to words present in the transcript that are proportional to a number of occurrences of a word in the string corresponding to the sentence vector and inversely proportional to a number of occurrences of the word in the transcript;
determine pairwise dot products of the respective sentence vectors;
determine respective scores for the first set of strings of the transcript based on the respective sentence vectors;
determine respective scores for a second set of strings of the transcript based on a number of remaining words, after the stop words are removed, being less than the threshold;
select a selected string for highlighting from the transcript based on respective scores of strings;
select a selected video excerpt from a video of the conference based on the respective timestamp of the selected string, wherein the selected video excerpt includes multiple frames of video; and
generate a video conference summary as a sequence of video excerpts from the video, including the selected video excerpt.
12 . The system of claim 11 , wherein a non-zero element of the respective sentence vector for one of the strings of the transcript is a term frequency-inverse document frequency for a word associated with the non-zero element.
13 . The system of claim 11 , wherein the memory stores instructions executable by the processor to:
generate a highlighted transcript as a copy of the transcript with a subset of the strings highlighted, wherein the selected string is highlighted;
present the highlighted transcript to a user;
receive user edits to the highlighting of the highlighted transcript; and
select the selected video excerpt from the video of the conference based on the respective timestamp of a string selected based on the user edits to the highlighting.
14 . The system of claim 11 , wherein the memory stores instructions executable by the processor to:
detect an action item phrase in a string from the transcript, wherein the selected string is selected based on presence of the action item phrase.
15 . The system of claim 11 , wherein the memory stores instructions executable by the processor to:
detect one or more words from a set of keywords in a string from the transcript, wherein the selected string is selected based on presence of the one or more words from the set of keywords.
16 . A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
obtaining a transcript of a conference, wherein the transcript includes strings with respective timestamps;
removing stop words from the strings of the transcript;
selecting, from amongst the strings of the transcript, a first set of strings of the transcript with a number of remaining words, after the stop words are removed, greater than a threshold for determination of respective sentence vectors;
determining respective sentence vectors for the first set of strings of the transcript, wherein a sentence vector has elements corresponding to words present in the transcript that are proportional to a number of occurrences of a word in the string corresponding to the sentence vector and inversely proportional to a number of occurrences of the word in the transcript;
determining respective scores for the first set of strings of the transcript based on the respective sentence vectors;
determining respective scores for a second set of strings of the transcript based on a number of remaining words, after the stop words are removed, being less than the threshold;
selecting a selected string for highlighting from the transcript based on respective scores of strings;
selecting a selected video excerpt from a video of the conference based on the respective timestamp of the selected string, wherein the selected video excerpt includes multiple frames of video; and
generating a video conference summary as a sequence of video excerpts from the video, including the selected video excerpt.
17 . The non-transitory computer-readable storage medium of claim 16 , wherein determining respective scores for the first set of strings of the transcript based on the respective sentence vectors comprises:
determining pairwise similarity scores of the respective sentence vectors, wherein one of the pairwise similarity scores is determined based one or more of a dot product, a Euclidean distance, a Pearson Correlation, a Jaccard coefficient, and a Tanimoto coefficient between two of the respective sentence vectors; and
determining the respective score for one of the strings based on a graph with vertices corresponding to the strings of the transcript and to edge weights corresponding to the pairwise similarity scores.
18 . The non-transitory computer-readable storage medium of claim 16 , comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
generating a highlighted transcript as a copy of the transcript with a subset of the strings highlighted, wherein the selected string is highlighted;
presenting the highlighted transcript to a user;
receiving user edits to the highlighting of the highlighted transcript; and
selecting the selected video excerpt from the video of the conference based on the respective timestamp of a string selected based on the user edits to the highlighting.
19 . The non-transitory computer-readable storage medium of claim 16 , wherein determining respective scores for the first set of strings of the transcript based on the respective sentence vectors comprises determining pairwise dot products of the respective sentence vectors, and determining the respective score for one of the strings in the first set of strings of the transcript based on a sum of the pairwise dot products for the respective sentence vector of the one of the strings in the first set of strings of the transcript.
20 . The system of claim 11 , wherein the memory stores instructions executable by the processor to:
determine the respective score for one of the strings based on a sum of the pairwise dot products for the respective sentence vector of the one of the strings.