Methods and systems for automatic discovery of fraudulent calls using speaker recognition

1. A computer-implemented method for identifying a voice associated with undesirable activity, comprising:

receiving an audio recording or audio stream associated with a voice;

determining a plurality of audio components of the audio recording or audio stream;

comparing the voice, using the determined plurality of audio components, with a library of voices, each voice in the library of voices defined by a cluster of audio recordings clustered together based on audio components of the audio recordings such that audio recordings associated with a same voice but different accounts are associated with a same cluster;

based on the comparing of the voice to the library of voices, identifying a cluster likely to correspond to a same voice as the voice associated with the received audio recording or audio stream;

determining, based on account information associated with the received audio recording or audio stream and metadata associating each audio recording in the identified cluster with a respective account, whether the voice associated with the identified cluster is associated with a plurality of accounts associated with different people; and

in response to determining that the identified cluster is associated with a plurality of accounts associated with different people, identifying the received audio recording or audio stream as associated with undesirable activity.

2. The computer-implemented method of claim 1 , further comprising:

adding the identified cluster to a voice blacklist.

3. The computer-implemented method of claim 1 , further comprising:

in response to determining that the received audio recording or audio stream is associated with undesirable activity, determining a genuine account associated with the received audio recording or audio stream, and one or more of:

determining a verified owner of the genuine account, and transmitting an alert to the verified owner;

adding one or more of a restriction or a verification requirement to the genuine account; or

determining an entity associated with the genuine account, and transmitting an alert to the entity.

4. The computer-implemented method of claim 1 , further comprising:

in response to determining that the received audio recording or audio stream is associated with undesirable activity, determining a genuine account associated with the voice associated with the received audio recording or audio stream; and

one or more of:

determining, based on the genuine account, a genuine identity of a fraudulent caller associated with the received audio recording or audio stream;

flagging the genuine account as associated with a fraudulent caller;

adding a restriction to the genuine account; or

transmitting information associated with the genuine account to a law enforcement entity.

5. The computer-implemented method of claim 1 , further comprising:

adding a further audio recording based on the received audio recording or audio stream to the library of voices; and

updating a clustering of the audio recordings in the library of voices.

6. The computer-implemented method of claim 5 , wherein updating the clustering of the audio recordings includes:

generating a respective multi-dimensional vector based on the respective plurality of audio components of each audio recording; and

comparing the respective multi-dimensional vectors.

7. The computer-implemented method of claim 6 , wherein comparing the respective multi-dimensional vectors includes one or more of:

determining a minimum similarity between each of the respective multi-dimensional vectors; or

determining an average similarity between each of the respective multi-dimensional vectors.

8. The computer-implemented method of claim 1 , wherein identifying the cluster likely to correspond to a same voice as the voice associated with the received audio recording or audio stream includes:

determining that the voice associated with the received audio recording or audio stream does not correspond to any of the clusters in the library of voices; and

generating a new cluster based on the received audio recording or audio stream.

9. The computer-implemented method of claim 1 , wherein identifying the received audio recording or audio stream as associated with undesirable activity is further performed in response to determining that the identified cluster is not included in a voice whitelist.

10. The computer-implemented method of claim 1 , wherein:

the library of voices represents each audio recording as an n-dimensional vector formed by the plurality of audio components of the audio recording; and

comparing the voice, using the determined plurality of audio components, includes:

representing the determined plurality of audio components as a further n-dimensional vector; and

performing a clustering operation of the further n-dimensional vector with the n-dimensional vectors of the library of voices.

11. A system for identifying a voice associated with undesirable activity, comprising:

at least one memory storing instructions and a library of voices, each voice in the library of voices defined by a cluster of audio recordings clustered together based on audio components of the audio recordings such that audio recordings associated with a same voice but different accounts are associated with a same cluster; and

at least one processor operatively connected to the at least one memory, and configured to execute the instructions to perform operations, including:

receiving an audio recording or audio stream associated with a voice;

determining a plurality of audio components of the audio recording or audio stream;

comparing the voice, using the determined plurality of audio components, with the library of voices;

based on the comparing of the voice to the library of voices, identifying a cluster likely to correspond to a same voice as the voice associated with the received audio recording or audio stream;

determining, based on account information associated with the received audio recording or audio stream and metadata associating each audio recording in the identified cluster with a respective account, whether the voice associated with the identified cluster is associated with a plurality of accounts associated with different people; and

in response to determining that the identified cluster is associated with a plurality of accounts associated with different people, identifying the received audio recording or audio stream as associated with undesirable activity.

12. The system of claim 11 , wherein the operations further include:

adding the identified cluster to a voice blacklist.

13. The system of claim 11 , wherein the operations further include:

in response to determining that the received audio recording or audio stream is associated with undesirable activity, determining a genuine account associated with the received audio recording or audio stream, and one or more of:

determining a verified owner of the genuine account, and transmitting an alert to the verified owner;

adding one or more of a restriction or a verification requirement to the genuine account; or

determining an entity associated with the genuine account, and transmitting an alert to the entity.

14. The system of claim 11 , wherein the operations further include:

in response to determining that the received audio recording or audio stream is associated with undesirable activity, determining a genuine account associated with the voice associated with the received audio recording or audio stream; and

one or more of:

determining, based on the genuine account, a genuine identity of a fraudulent caller associated with the received audio recording or audio stream;

flagging the genuine account as associated with a fraudulent caller;

adding a restriction to the genuine account; or

transmitting information associated with the genuine account to a law enforcement entity.

15. The system of claim 11 , wherein the operations further include:

adding a further audio recording based on the received audio recording or audio stream to the library of voices; and

updating a clustering of the audio recordings in the library of voices.

16. The system of claim 15 , wherein updating the clustering of the audio recordings includes:

generating a respective multi-dimensional vector based on the respective plurality of audio components of each audio recording; and

comparing the respective multi-dimensional vectors by one or more of:

determining a minimum similarity between each of the respective multi-dimensional vectors; or

determining an average similarity between each of the respective multi-dimensional vectors.

17. The system of claim 11 , wherein identifying the cluster likely to correspond to a same voice as the voice associated with the received audio recording or audio stream includes:

determining that the voice associated with the received audio recording or audio stream does not correspond to any of the clusters in the library of voices; and

generating a new cluster based on the received audio recording or audio stream.

18. The system of claim 11 , wherein identifying the received audio recording or audio stream as associated with undesirable activity is further performed in response to determining that the identified cluster is not included in a voice whitelist.

19. The system of claim 11 , wherein:

the library of voices represents each audio recording as an n-dimensional vector formed by the plurality of audio components of the audio recording; and

comparing the voice, using the determined plurality of audio components, includes:

representing the determined plurality of audio components as a further n-dimensional vector; and

performing a clustering operation of the further n-dimensional vector with the n-dimensional vectors of the library of voices.

20. A computer-implemented method for identifying a voice associated with undesirable activity, comprising:

receiving an audio recording or audio stream associated with a voice;

determining a plurality of audio components of the audio recording or audio stream;

comparing the voice, using the determined plurality of audio components, with a library of voices, each voice in the library of voices defined by a cluster of audio recordings clustered together based on audio components of the audio recordings such that audio recordings associated with a same voice but different accounts are associated with a same cluster;

based on the comparing of the voice to the library of voices, identifying a cluster likely to correspond to a same voice as the voice associated with the received audio recording or audio stream;

determining, based on account information associated with the received audio recording or audio stream and metadata associating each audio recording in the identified cluster with a respective account, whether the voice associated with the identified cluster is associated with a plurality of accounts associated with different people;

in response to determining that the identified cluster is associated with a plurality of accounts associated with different people, identifying the received audio recording or audio stream as associated with undesirable activity;

adding a further audio recording based on the received audio recording or audio stream to the identified cluster; and

adding the identified cluster to a voice blacklist.