Transcript alignment
View Patent ↗An approach to alignment of transcripts with recorded audio is tolerant of moderate transcript inaccuracies, untranscribed speech, and significant non-speech noise. In one aspect, a number of search terms are formed from the transcript such that each search term is associated with a location within the transcript. Possible locations of the search terms are then determined in the audio recording. The audio recording and the transcript are then aligned using the possible locations of the search terms. In another aspect a search expression is accepted, and then a search is performed for spoken occurrences of the search expression in an audio recording. This search includes searching for text occurrences of the search expression in a text transcript of the audio recording, and searching for spoken occurrences of the search expression in the audio recording.
1. A method for aligning an audio recording and a transcript comprising:
forming a plurality of search terms from the transcript, each search term associated with a location within the transcript;
determining zero or more putative locations of each of the search terms in a time interval of the audio recording, including for at least some search terms determining multiple putative locations in the time interval of the audio recording; and
after determining the putative locations, aligning the time interval of the audio recording and the transcript using the determined putative locations of the search terms, including, for at least some of the search terms, selecting one of the putative locations of the search term for aligning with the location within the transcript that is associated with the search term.
2. The method of claim 1 wherein forming the search terms includes forming one or more search terms for each of a plurality of segments of the transcript.
3. The method of claim 2 wherein forming the search terms for each of the segments includes forming one or more search terms for each of a plurality of text lines of the transcript.
4. The method of claim 1 wherein determining putative locations of the search terms includes applying a word spotting approach to determine one or more putative locations for each of the search terms.
5. The method of claim 1 wherein determining the putative locations of the search terms includes associating each of the putative locations with a score characterizing a quality of match of the search term and the corresponding putative location.
6. The method of claim 1 further comprising packaging the audio recording, the transcript, and data representing the alignment of the audio recording and the transcript on one or more computer-readable storage devices.
7. The method of claim 6 further comprising storing software on the one or more computer-readable storage devices comprising processor code for accessing the audio recording according to the alignment of the audio recording and the transcript.
8. The method of claim 1 wherein the transcript includes speaker indications, and aligning the audio recording and the transcript includes using speaker characteristics in the audio recording to match the speaker indications.
9. The method of claim 1 wherein the audio recording is associated with a video recording and the transcript includes indications of video events, and aligning the audio recording and the transcript includes using video characteristics in the associated video recording to match the video event.
10. The method of claim 1 wherein the audio recording includes a recording of a court session, and the transcript includes a court transcript.
11. The method of claim 1 wherein the audio recording includes audio associated with a television program, and the transcript includes a closed captioning text for the television program.
12. The method of claim 1 wherein the audio recording includes air-traffic control recordings, and the transcript includes an air-traffic control transcript.
13. The method of claim 1 wherein the audio recording includes audio associated with a performance, and the transcript includes a script used for the performance.
14. The method of claim 1 wherein the time interval of the audio recording comprises the audio recording in its entirety.
15. The method of claim 1 wherein the aligning comprises using a gap alignment.
16. The method of claim 1 wherein the aligning comprises using an optimization alignment.
17. The method of claim 1 wherein the aligning comprises using a blind alignment.
18. The method of claim 1 wherein aligning comprises aligning the time interval of the audio recording and the transcript using the determined putative locations of the search terms until all of the search terms are marked as aligned, and the aligning further comprises:
dividing the time interval of the audio recording into a plurality of second time intervals; and
performing the aligning on each of the plurality of second time intervals.
19. The method of claim 1 wherein determining zero or more putative locations of each of the search terms includes not determining any putative locations of at least some of the search terms in the time interval.
20. The method of claim 19 wherein aligning the time interval of the audio recording and the transcript includes omitting portions of the transcript from the alignment.
21. The method of claim 1 wherein aligning the time interval of the audio recording and the transcript includes aligning an incomplete transcript of the audio recording.
22. The method of claim 1 wherein aligning the time interval of the audio recording and the transcript includes aligning a transcript that includes text that does not reflect words spoken in the audio recording.
23. The method of claim 22 wherein aligning the time interval of the audio recording and the transcript includes discarding at least some of the search terms.