Systems and methods for a two pass diarization, automatic speech recognition, and transcript generation
In one embodiment, a method for transcript generation includes receiving an audio file and dividing it into a plurality of chunks. The method further includes sending each instance of the plurality of chunks to a speech service module. The method further includes converting speech to text for each instance of the plurality of chunks and returning the text for each instance of the plurality of chunks. The method further includes merging the text for each instance of the plurality of chunks to yield an audio file transcript and sending the audio file and chunks to a diarization module. The method further includes performing first pass diarization on the chunks to yield a plurality of diarized chunks and performing second pass diarization on the plurality of diarized chunks and the audio file to yield a diarized audio file. The method further includes merging the files to yield a final transcript.
1. A method of performing diarization on a sound recording, the method comprising:
receiving a sound recording;
breaking the sound recording into a plurality of chunks;
performing a first diarization on the plurality of chunks, wherein the performing the first diarization on the plurality of chunks occurs simultaneously, and wherein the performing includes breaking each of the plurality of chunks into a plurality of segments, for each of the plurality of segments generating statistical speaker information descriptive of the sound characteristics in that segment, and clustering, within each chunk of the plurality of chunks, segments having similar statistical speaker information to generate within each chunk of the plurality of chunks groups of segments grouped according to the similar statistical speaker information;
performing a second diarization by clustering between the plurality of chunks, the groups of segments according to grouped similar statistical speaker information, the grouped similar statistical speaker information being characteristics of speech of each group for the groups of segments, wherein the second diarization performs a modified I-Vector scoring, based on the groups of segments according to grouped similar statistical speaker information, I-vectors of the groups of segments according to grouped similar statistical speaker information are averaged and then compared to other averaged I-vectors, where a closeness of two or more averaged I-vectors is compared accordingly clustered based on similarity;
creating a new i-vector for the groups of segments according to grouped similar statistical speaker information.
2. The method of claim 1 , further comprising:
transcoding the sound recording according to a known codec.
3. The method of claim 1 , further comprising:
creating a sound recording transcript from the sound recording;
sending the sound recording transcript to a post process module;
applying punctuation and casing to the sound recording transcript.
4. The method of claim 1 , wherein the speaker identification information is an I-vector.
5. The method of claim 1 , wherein the second diarization includes giving each of a plurality of speakers for each of the plurality of diarized chunks a unique identifier.
6. The method of claim 5 , wherein the second diarization includes, for associated segments of the plurality of segments for each unique identifier, averaging the speaker identification information of the associated segments to yield averaged speaker identification information.
7. The method of claim 6 , wherein the second diarization includes, assigning identified segments of the plurality of segments from all of the plurality of chunks a final speaker based on correlation between the averaged speaker identification information for the associated segments of the plurality of segments for each unique identifier.
8. The method of claim 3 , further comprising: creating a final transcript from the sound recording transcript; and outputting a final transcript in a fixed and tangible format.
9. A system for performing diarization on a sound recording, the system comprising:
a diarization module configured to
receive a sound recording;
break the sound recording into a plurality of chunks;
perform a first diarization on the plurality of chunks, wherein the first diarization on the plurality of chunks occurs simultaneously, and wherein the performing includes breaking each of the plurality of chunks into a plurality of segments, for each of the plurality of segments generating statistical speaker information descriptive of the sound characteristics in that segment, and clustering, within each chunk of the plurality of chunks, segments having similar statistical speaker information to generate within each chunk of the plurality of chunks groups of segments grouped according to the similar statistical speaker information;
perform a second diarization by clustering between the plurality of chunks, the groups of segments according to grouped similar statistical speaker information, the grouped similar statistical speaker information being characteristics of speech of each group for the groups of segments, wherein the second diarization performs a modified I-Vector scoring, based on the groups of segments according to grouped similar statistical speaker information, I-vectors of the groups of segments according to grouped similar statistical speaker information are averaged and then compared to other averaged I-vectors, where a closeness of two or more averaged I-vectors is compared accordingly clustered based on similarity;
creating a new i-vector for the groups of segments according to grouped similar statistical speaker information.
10. The system of claim 9 , wherein the diarization module is further configured to transcode the sound recording according to a known codec.
11. The method of claim 9 , wherein the diarization module is further configured to
create a sound recording transcript from the sound recording;
send the sound recording transcript to a post process module;
apply punctuation and casing to the sound recording transcript.
12. The method of claim 9 , wherein the speaker identification information is an I-vector.
13. The method of claim 9 , wherein the second diarization includes giving each of a plurality of speakers for each of the plurality of diarized chunks a unique identifier.
14. The method of claim 13 , wherein the second diarization includes, for associated segments of the plurality of segments for each unique identifier, averaging the speaker identification information of the associated segments to yield averaged speaker identification information.
15. The method of claim 14 , wherein the second diarization includes, assigning identified segments of the plurality of segments from all of the plurality of chunks a final speaker based on correlation between the averaged speaker identification information for the associated segments of the plurality of segments for each unique identifier.
16. The method of claim 11 , wherein the diarization module is further configured to create a final transcript from the sound recording transcript; and outputting a final transcript in a fixed and tangible format.
17. The method of claim 1 , wherein the groups of segments grouped according to the similar statistical speaker information are clustered as speakers.
18. The method of claim 9 , wherein the groups of segments grouped according to the similar statistical speaker information are clustered as speakers.