Online speaker diarization using local and global clustering
A method includes obtaining at least a portion of an audio stream containing speech activity. At least the portion of the audio stream includes multiple segments. The method also includes, for each of the segments, generating an embedding vector that represents the segment. The method further includes, within each of multiple local windows, clustering the embedding vectors into one or more clusters to perform speaker identification. Different clusters correspond to different speakers. The method also includes presenting at least one first sequence of speaker identities based on the speaker identification for the local windows. The method further includes, within each of multiple global windows, clustering the embedding vectors into one or more clusters to perform speaker identification. Each global window includes two or more of the local windows. The method further includes presenting at least one second sequence of speaker identities based on the speaker identification for the global windows.
1. A method for performing online speaker diarization, the method comprising:
obtaining at least a portion of an audio stream containing speech activity, at least the portion of the audio stream comprising multiple segments;
for each of the multiple segments, generating an embedding vector that represents the segment;
within each of multiple local windows, performing monotonically-increasing chunk-based spectral clustering of the embedding vectors into one or more clusters to perform speaker identification, wherein different clusters correspond to different speakers, each local window having more than one of the segments;
presenting at least one first sequence of speaker identities based on the speaker identification performed for the local windows;
within each of multiple global windows, clustering the embedding vectors into one or more clusters to perform speaker identification, wherein each global window includes two or more of the local windows; and
presenting at least one second sequence of speaker identities based on the speaker identification performed for the global windows;
wherein performing the monotonically-increasing chunk-based spectral clustering comprises clustering different subsets of the embedding vectors in the local window such that each subsequent subset includes (i) the embedding vectors of at least one prior subset and (ii) at least one additional embedding vector not included in the at least one prior subset.
2. The method of claim 1 , further comprising:
matching entries in different sequences of speaker identities; and
replacing at least some of the entries in one sequence of speaker identities with at least some of the entries in another sequence of speaker identities so that clusters generated in different windows correspond to common speakers.
3. The method of claim 1 , wherein the local windows partially overlap one another.
4. The method of claim 1 , wherein:
the global windows comprise first and second global windows; and
the second global window is larger than and includes the first global window.
5. The method of claim 1 , wherein a final one of the global windows encompasses an entirety of the audio stream.
6. The method of claim 1 , further comprising:
determining a similarity between clusters of the embedding vectors in different ones of the local windows; and
in response to determining that the similarity exceeds a threshold, using a common cluster identifier for the clusters of the embedding vectors in the different local windows.
7. An apparatus comprising:
at least one processing device configured to:
obtain at least a portion of an audio stream containing speech activity, at least the portion of the audio stream comprising multiple segments;
for each of the multiple segments, generate an embedding vector that represents the segment;
within each of multiple local windows, perform monotonically-increasing chunk-based spectral clustering of the embedding vectors into one or more clusters to perform speaker identification, wherein different clusters correspond to different speakers, each local window having more than one of the segments;
initiate presentation of at least one first sequence of speaker identities based on the speaker identification performed for the local windows;
within each of multiple global windows, cluster the embedding vectors into one or more clusters to perform speaker identification, wherein each global window includes two or more of the local windows; and
initiate presentation of at least one second sequence of speaker identities based on the speaker identification performed for the global windows;
wherein, to perform the monotonically-increasing chunk-based spectral clustering, the at least one processing device is configured to cluster different subsets of the embedding vectors in the local window such that each subsequent subset includes (i) the embedding vectors of at least one prior subset and (ii) at least one additional embedding vector not included in the at least one prior subset.
8. The apparatus of claim 7 , wherein the at least one processing device is further configured to:
match entries in different sequences of speaker identities; and
replace at least some of the entries in one sequence of speaker identities with at least some of the entries in another sequence of speaker identities so that clusters generated in different windows correspond to common speakers.
9. The apparatus of claim 7 , wherein the local windows partially overlap one another.
10. The apparatus of claim 7 , wherein:
the global windows comprise first and second global windows; and
the second global window is larger than and includes the first global window.
11. The apparatus of claim 7 , wherein a final one of the global windows encompasses an entirety of the audio stream.
12. The apparatus of claim 7 , wherein the at least one processing device is further configured to:
determine a similarity between clusters of the embedding vectors in different ones of the local windows; and
in response to determining that the similarity exceeds a threshold, use a common cluster identifier for the clusters of the embedding vectors in the different local windows.
13. A non-transitory computer readable medium containing instructions that when executed cause at least one processor to:
obtain at least a portion of an audio stream containing speech activity, at least the portion of the audio stream comprising multiple segments;
for each of the multiple segments, generate an embedding vector that represents the segment;
within each of multiple local windows, perform monotonically-increasing chunk-based spectral clustering of the embedding vectors into one or more clusters to perform speaker identification, wherein different clusters correspond to different speakers, each local window having more than one of the segments;
initiate presentation of at least one first sequence of speaker identities based on the speaker identification performed for the local windows;
within each of multiple global windows, cluster the embedding vectors into one or more clusters to perform speaker identification, wherein each global window includes two or more of the local windows; and
initiate presentation of at least one second sequence of speaker identities based on the speaker identification performed for the global windows;
wherein the instructions that when executed cause the at least one processor to perform the monotonically-increasing chunk-based spectral clustering comprise instructions that when executed cause the at least one processor to cluster different subsets of the embedding vectors in the local window such that each subsequent subset includes (i) the embedding vectors of at least one prior subset and (ii) at least one additional embedding vector not included in the at least one prior subset.
14. The non-transitory computer readable medium of claim 13 , further containing instructions that when executed cause the at least one processor to:
match entries in different sequences of speaker identities; and
replace at least some of the entries in one sequence of speaker identities with at least some of the entries in another sequence of speaker identities so that clusters generated in different windows correspond to common speakers.
15. The non-transitory computer readable medium of claim 13 , wherein the local windows partially overlap one another.
16. The non-transitory computer readable medium of claim 13 , wherein:
the global windows comprise first and second global windows; and
the second global window is larger than and includes the first global window.
17. The non-transitory computer readable medium of claim 13 , further containing instructions that when executed cause the at least one processor to:
determine a similarity between clusters of the embedding vectors in different ones of the local windows; and
in response to determining that the similarity exceeds a threshold, use a common cluster identifier for the clusters of the embedding vectors in the different local windows.
18. The method of claim 1 , wherein portions of processing during the clustering of the embedding vectors in the local windows and portions of processing during the clustering of the embedding vectors in the global windows occur in parallel.
19. The apparatus of claim 7 , wherein the at least one processing device is configured to perform portions of processing during the clustering of the embedding vectors in the local windows and portions of processing during the clustering of the embedding vectors in the global windows in parallel.
20. The non-transitory computer readable medium of claim 13 , wherein the instructions when executed cause the at least one processor to perform portions of processing during the clustering of the embedding vectors in the local windows and portions of processing during the clustering of the embedding vectors in the global windows in parallel.