Transcript paragraph segmentation and visualization of transcript paragraphs
Embodiments of the present invention provide systems, methods, and computer storage media for segmenting a transcript into paragraphs. In an example embodiment, a transcript is segmented to start a new paragraph whenever there is a change in speaker and/or a long pause in speech. If any remaining paragraphs are longer than a designated length or duration (e.g., 50 or 100 words), each of those paragraphs is segmented using dynamic programming to minimize a cost function that penalizes candidate paragraphs based on divergence from a target paragraph length and/or that rewards candidate paragraphs that group semantically similar sentences. As such, the transcript is visualized, segmented at the identified paragraphs.
1. One or more computer storage media storing computer-useable instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:
causing generation of a representation of a paragraph segmentation of a transcript that segments one or more transcript paragraphs into smaller transcript paragraphs by using dynamic programming to minimize a cost function based on a target paragraph length and semantic coherency of text within one or more paragraphs of the paragraph segmentation; and
causing a user interface to present a visualization of the paragraph segmentation of the transcript.
2. The one or more computer storage media of claim 1 , wherein the transcript is of an audio track, and the generation of the representation of the paragraph segmentation is based on speech pauses detected in the audio track using an audio classifier.
3. The one or more computer storage media of claim 1 , wherein the generation of the representation of the paragraph segmentation is based on speech pauses detected using differences in word or sentence timings extracted from the transcript.
4. The one or more computer storage media of claim 1 , wherein the generation of the representation of the paragraph segmentation comprises identifying the one or more transcript paragraphs that are longer than a designated length or duration and, for each identified transcript paragraph, segmenting the identified transcript paragraph using dynamic programming to minimize the cost function.
5. The one or more computer storage media of claim 1 , wherein the generation of the representation of the paragraph segmentation comprises using dynamic programming to minimize the cost function that penalizes candidate segmentations that include candidate paragraphs with detected speech pauses longer than a target length or duration.
6. The one or more computer storage media of claim 1 , wherein the generation of the representation of the paragraph segmentation comprises using dynamic programming to minimize the cost function that rewards candidate paragraphs that group semantically similar sentences.
7. The one or more computer storage media of claim 1 , wherein the generation of the representation of the paragraph segmentation comprises quantifying the semantic coherency by generating a measure of similarity of each pair of sentences in a candidate paragraph and combining the measure of similarity for each pair to generate a measure of paragraph similarity for the candidate paragraph.
8. The one or more computer storage media of claim 1 , wherein the paragraph segmentation segments the transcript at a subset of sentence boundaries identified from the transcript.
9. The one or more computer storage media of claim 1 , wherein the transcript is of an audio track of a video, the user interface is a transcript interface of a video editing application, and the transcript interface is configured to interpret input selecting transcript text, via an interaction with the visualization of the paragraph segmentation of the transcript, as an instruction to select a corresponding video segment of the video.
10. A method comprising:
triggering generation of a representation of a paragraph segmentation of a transcript that segments one or more transcript paragraphs into smaller transcript paragraphs by using dynamic programming to minimize a cost function based on a target paragraph length and semantic coherency; and
causing a user interface to present transcript text of the transcript using the paragraph segmentation.
11. The method of claim 10 , wherein the transcript is of an audio track, and the generation of the representation of the paragraph segmentation is based on speech pauses detected in the audio track using an audio classifier.
12. The method of claim 10 , wherein the generation of the representation of the paragraph segmentation is based on speech pauses detected using differences in word or sentence timings extracted from the transcript.
13. The method of claim 10 , wherein the generation of the representation of the paragraph segmentation comprises identifying the one or more transcript paragraphs that are longer than a designated length or duration and, for each identified transcript paragraph, segmenting the identified transcript paragraph using dynamic programming to minimize the cost function.
14. The method of claim 10 , wherein the generation of the representation of the paragraph segmentation comprises using dynamic programming to minimize the cost function that penalizes candidate segmentations that include candidate paragraphs with detected speech pauses longer than a designated length or duration.
15. The method of claim 10 , wherein the generation of the representation of the paragraph segmentation comprises using dynamic programming to minimize the cost function that rewards candidate paragraphs that group semantically similar sentences.
16. The method of claim 10 , wherein the generation of the representation of the paragraph segmentation comprises quantifying the semantic coherency by generating a measure of similarity of each pair of sentences in a candidate paragraph and combining the measure of similarity for each pair to generate a measure of paragraph similarity for the candidate paragraph.
17. The method of claim 10 , wherein the paragraph segmentation segments the transcript at a subset of sentence boundaries identified from the transcript.
18. The method of claim 10 , wherein the transcript is of an audio track of a video, the user interface is a transcript interface of a video editing application, and the transcript interface is configured to interpret input selecting a portion of the transcript text as an instruction to select a corresponding video segment of the video.
19. A computer system comprising one or more processors and memory configured to provide computer program instructions to the one or more processors, the computer program instructions comprising:
a video interaction engine configured to trigger generation of a representation of a paragraph segmentation of a transcript of a video that segments one or more transcript paragraphs into smaller transcript paragraphs by using dynamic programming to minimize a cost function based on a target paragraph length and semantic coherency; and
a transcript tool configured to cause presentation of transcript text of the transcript using the paragraph segmentation.
20. The computer system of claim 19 , wherein the video interaction engine and the transcript tool are part of a video editing application, and the transcript tool is configured to interpret input selecting a portion of the transcript text as an instruction to select a corresponding video segment of the video.