Methods and systems for automatic discovery of fraudulent calls using speaker recognition
A method for determining potentially undesirable voices, in embodiments, includes: receiving audio recordings comprising voices associated with undesirable activity, and determining audio components of each of the audio recordings. The method may further comprise generating a multi-dimensional vector of the audio components for each of the plurality of audio recordings, and comparing audio components between the multi-dimensional vectors to determine clusters of multi-dimensional vectors, each cluster comprising two or more of the multi-dimensional vectors of audio components, wherein each cluster corresponds to a blacklisted voice. The method may further comprise receiving an audio recording or audio stream, and determining whether the audio recording or audio stream is associated with a voice associated with undesirable activity based on a comparison to the clusters.
1 . A computer-implemented method for identifying a voice associated with undesirable activity, comprising:
obtaining voiceprint data that includes vector representations of voices from a plurality of audio recordings;
identifying one or more vector representations that are unlikely to cluster with others of the vector representations;
generating a sub-set of vector representations by discarding the one or more identified vector representations that are unlikely to cluster with others of the vector representations;
grouping the sub-set of vector representations into a plurality of clusters by performing a clustering operation:
identifying one or more spurious clusters by performing a filtering operation on the plurality of clusters; and
forming a voiceprint library for identifying a voice associated with undesirable activity by discarding the one or more spurious clusters from the plurality of clusters.
2 . The computer-implemented method of claim 1 , wherein the clustering operation includes iteratively, until no grouping of one or more vector representations has a similarity above a predetermined threshold, performing the operations of:
determining a similarity between each grouping and one or more neighboring groupings; and
merging groupings that have a similarity above the predetermined threshold.
3 . The computer-implemented method of claim 2 , wherein the similarity between each grouping and the one or more neighboring groupings is defined by a complete linkage criterion between all vector representations in the groupings to be merged.
4 . The computer-implemented method of claim 2 , wherein the similarity between each grouping and the one or more neighboring groupings is defined by an average linkage criterion between the groupings to be merged.
5 . The computer-implemented method of claim 1 , wherein the clustering operation includes an agglomerative hierarchical clustering process.
6 . The computer-implemented method of claim 1 , wherein:
the voiceprint data further includes metadata associated with the vector representations; and
one or more of the identifying, grouping, or forming is further based on the metadata.
7 . The computer-implemented method of claim 6 , wherein the metadata includes information regarding text corresponding to at least a portion of speech from a corresponding audio recording.
8 . The computer-implemented method of claim 7 , wherein the information regarding text includes a frequency count for each word in a predetermined set of words.
9 . The computer-implemented method of claim 6 , wherein the metadata includes account information associated with a corresponding audio recording.
10 . The computer-implemented method of claim 1 , wherein the filtering operation is based on one or more of cluster size, cluster coherence, metadata associated with vector representations within each cluster, or a proportion of vector representations within each cluster that have a predetermined association with undesirable activity.
11 . The computer-implemented method of claim 1 , wherein the filtering operation is configured to identify one or more clusters that only contain vector representations associated with a single account, such that forming the voiceprint library includes discarding clusters only associated with a single account.
12 . The computer-implemented method of claim 1 , further comprising:
configuring the voiceprint library as a blacklist, such that a future audio recording is compared against each cluster in the voiceprint library, the voiceprint library configured to blacklist the future audio recording upon a similarity between the audio recording and at least one of the clusters being above a predetermined similarity threshold.
13 . A computer-implemented method for identifying a voice associated with undesirable activity, comprising:
obtaining voiceprint data that includes vector representations of voices from a plurality of audio recordings;
performing a nearest-neighbor analysis on the vector representations to identify one or more vector representations having a similarity below a predetermined threshold;
generating a sub-set of vector representations by discarding the one or more identified vector representations;
grouping the sub-set of vector representations into a plurality of clusters by performing a clustering operation:
identifying one or more spurious clusters by performing a filtering operation on the plurality of clusters, wherein the one or more clusters are identified as spurious based on one or more of cluster size, cluster coherence, number of voices represented by vectors within a same cluster, or number of accounts associated with vector representations within a same cluster; and
forming a voiceprint library for identifying a voice associated with undesirable activity by discarding the one or more spurious clusters from the plurality of clusters.
14 . The computer-implemented method of claim 13 , wherein the nearest-neighbor analysis is an exact nearest-neighbor analysis.
15 . The computer-implemented method of claim 13 , wherein the nearest-neighbor analysis is an approximate nearest-neighbor analysis.
16 . The computer-implemented method of claim 13 , wherein the similarity for each vector representation is based on a similarity score for the vector representation and each of a predetermined number of nearest neighbors being below the predetermined threshold.
17 . The computer-implemented method of claim 13 , wherein the similarity includes one or more of a cosine similarity or a log-likelihood ratio.
18 . The computer-implemented method of claim 13 , wherein the performing of the nearest-neighbor analysis and the generating of the sub-set of vector representations is performed in response to determining that a total quantity of the vector representations is above a predetermined quantity threshold.
19 . The computer-implemented method of claim 13 , wherein the performing of the nearest-neighbor analysis and the generating of the sub-set of vector representations is performed in response to determining that a predicted time to perform the clustering operation on the vector representations is above a predetermined time threshold.
20 . A computer-implemented method for identifying a voice associated with undesirable activity, comprising:
obtaining voiceprint data that includes vector representations of voices from a plurality of audio recordings;
identifying one or more vector representations that are unlikely to cluster with others of the vector representations by performing a nearest-neighbor analysis on the vector representations to identify one or more vector representations having a similarity below a predetermined threshold;
generating a sub-set of vector representations by discarding the one or more identified vector representations that are unlikely to cluster with others of the vector representations;
grouping the sub-set of vector representations into a plurality of clusters by performing a clustering operation, wherein the clustering operation includes iteratively, until no grouping of one or more vector representations has a similarity above a predetermined threshold, performing the operations of:
determining a similarity between each grouping and one or more neighboring groupings; and
merging groupings that have a similarity above the predetermined threshold:
identifying one or more spurious clusters by performing a filtering operation on the plurality of clusters, wherein a spurious cluster is a cluster that one or more of:
(i) includes vector representations of more than one voice;
(ii) includes vector representations that were clustered together due to common background noise or interference in corresponding audio recordings;
(iii) has a cluster coherence below a predetermined threshold;
(iv) includes less than a threshold number of vector representations; or
(v) includes vector representations associated with only one account;
forming a voiceprint library for identifying a voice associated with undesirable activity by discarding the one or more spurious clusters from the plurality of clusters; and
configuring the voiceprint library as a blacklist, such that a future audio recording is compared against each cluster in the voiceprint library, the voiceprint library configured to blacklist the future audio recording upon a similarity between the audio recording and at least one of the clusters being above a predetermined similarity threshold.