Method and system for user voice identification using ensembled deep learning algorithms

1. A method for training a user detection model to identify a user of a software application based on voice recognition, comprising:

receiving a data set including a plurality of recordings of voice interactions with users of a software application;

generating, for each respective recording in the data set, a spectrogram representation based on the respective recording, wherein the spectrogram representation is normalized with respect to a minimum amplitude and a maximum amplitude;

training a plurality of voice recognition models, wherein each model of the plurality of voice recognition models is trained based on the spectrogram representation for each of the plurality of recordings in the data set;

selecting, for a selected speaker of a plurality of speakers, an evaluation set of recordings;

identifying a similar speaker to the selected speaker by:

providing inputs based on the evaluation set of recordings to one or more of the plurality of voice recognition models, and

receiving an output from the one or more of the plurality of voice recognition models identifying the similar speaker as the selected speaker;

re-training the one or more of the plurality of voice recognition models based on a mapping of the selected speaker to the identified similar speaker; and

deploying the plurality of voice recognition models to an interactive voice response system.

2. The method of claim 1 , wherein the plurality of voice recognition models comprise:

a long-short term memory (LSTM) model;

a convolutional neural network (CNN) model; and

a temporal convolutional network (TCN) model.

3. The method of claim 1 , further comprising:

trimming each respective recording in the data set to remove silent periods in each respective voice recording; and

normalizing each trimmed recording such that a volume of the plurality of recordings in the data set has a similar loudness.

4. The method of claim 1 , further comprising:

receiving a clean data set including a plurality of clean recordings;

generating a noisy data set by adding random noise to each of the plurality of clean recordings in the clean data set; and

training a noise cancellation model based on the clean data set and the noisy data set.

5. The method of claim 4 , further comprising:

generating a plurality of noise-reduced recordings by processing each of the plurality of recordings in the data set through the trained noise cancellation model,

wherein the spectrogram representation generated for each respective recording is generated based on a noise-reduced recording associated with the respective recording.

6. The method of claim 1 , wherein training the plurality of voice recognition models comprises:

randomly selecting a number of speakers and a number of utterances for each speaker from the received data set;

generating a vector having dimensions based on the number of speakers, the number of utterances for each speaker, and a number of embedding dimensions;

generating a similarity matrix from the vector based on a calculated similarity between each pair of utterances selected from the data set; and

generating a predictive vector from the generated similarity matrix.

7. The method of claim 6 , wherein the predictive vector includes, for each entry in the similarity matrix, an indication of a speaker predicted to be associated with each of the utterances.

8. The method of claim 7 , wherein training the plurality of voice recognition models comprises calculating a loss based on a cross-entropy between the predictive vector and a ground truth vector identifying a speaker associated with each of the utterances.

9. The method of claim 1 , further comprising:

for each respective user of the software application, generating an embedding vectors using each of the plurality of voice recognition models;

generating a lookup table associating each of the generated embedding vectors with an identifier of the respective user; and

deploying the generated lookup table to the interactive voice response system.

10. A system, comprising:

a memory comprising executable instructions; and

a processor configured to execute the executable instructions and cause the system to:

receive a data set including a plurality of recordings of voice interactions with users of a software application;

generate, for each respective recording in the data set, a spectrogram representation based on the respective recording, wherein the spectrogram representation is normalized with respect to a minimum amplitude and a maximum amplitude;

train a plurality of voice recognition models, wherein each model of the plurality of voice recognition models is trained based on the spectrogram representation for each of the plurality of recordings in the data set;

select, for a selected speaker of a plurality of speakers, an evaluation set of recordings;

identify a similar speaker to the selected speaker by:

providing inputs based on the evaluation set of recordings to one or more of the plurality of voice recognition models, and

receiving an output from the one or more of the plurality of voice recognition models identifying the similar speaker as the selected speaker;

re-train the one or more of the plurality of voice recognition models based on a mapping of the selected speaker to the identified similar speaker; and

deploy the plurality of voice recognition models to an interactive voice response system.

11. The system of claim 10 , wherein the processor is further configured to cause the system to:

trim each respective recording in the data set to remove silent periods in each respective voice recording; and

normalize each trimmed recording such that a volume of the plurality of recordings in the data set has a similar loudness.

12. The system of claim 10 , wherein the processor is further configured to cause the system to:

receive a clean data set including a plurality of clean recordings;

generate a noisy data set by adding random noise to each of the plurality of clean recordings in the clean data set;

train a noise cancellation model based on the clean data set and the noisy data set; and

generate a plurality of noise-reduced recordings by processing each of the plurality of recordings in the data set through the trained noise cancellation model,

wherein the spectrogram representation generated for each respective recording is generated based on a noise-reduced recording associated with the respective recording.

13. The system of claim 10 , wherein the processor is configured to train the plurality of voice recognition models by:

randomly selecting a number of speakers and a number of utterances for each speaker from the received data set;

generating a vector having dimensions based on the number of speakers, the number of utterances for each speaker, and a number of embedding dimensions;

generating a similarity matrix from the vector based on a calculated similarity between each pair of utterances selected from the data set; and

generating a predictive vector from the generated similarity matrix.

14. The system of claim 10 , wherein the processor is further configured to cause the system to:

generate, for each respective user of the software application, an embedding vectors using each of the plurality of voice recognition models;

generate a lookup table associating each of the generated embedding vectors with an identifier of the respective user; and

deploy the generated lookup table to the interactive voice response system.

15. A method for identifying users of a software application based on voice recognition, comprising:

receiving a recording of a user utterance from a live support session with a user of the software application;

generating a feature vector representative of the user utterance based on a spectrogram representation of the received recording, wherein the spectrogram representation is normalized with respect to a minimum amplitude and a maximum amplitude;

generating a plurality of embedding vectors for the user utterance by processing the generated feature vector through a plurality of trained voice recognition models, wherein the plurality of trained voice recognition models was trained through a process comprising:

selecting, for a selected speaker of a plurality of speakers, an evaluation set of recordings;

identifying a similar speaker to the selected speaker by:

providing inputs based on the evaluation set of recordings to one or more of the plurality of trained voice recognition models, and

receiving an output from the one or more of the plurality of trained voice recognition models identifying the similar speaker as the selected speaker;

re-training the one or more of the plurality of trained voice recognition models based on a mapping of the selected speaker to the identified similar speaker;

selecting a plurality of users from a database of users based on a match between source contact information associated with the live support session and contact information included in user profiles for users of the software application;

calculating, for each respective user of a plurality of users, a similarity score between embedding vectors associated with the respective user and the generated plurality of embedding vectors for the user utterance; and

identifying a matching user from the plurality of users as a user of the plurality of users having a highest calculated similarity score; and

routing the live support session to a support agent, wherein the live support session includes information identifying the matching user.

16. The method of claim 15 , wherein the plurality of voice recognition models comprise:

a long-short term memory (LSTM) model;

a convolutional neural network (CNN) model; and

a temporal convolutional network (TCN) model.

17. The method of claim 15 , wherein:

each model of the plurality of trained voice recognition models is associated with a weight, and

calculating the similarity score between embedding vectors comprises weighting a similarity score between one of the plurality of embedding vectors generated by the model for the user utterance and one of the plurality of embedding vectors generated by the model for the respective user by the weight associated with the model.

18. The method of claim 15 , wherein generating the feature vector representative of the user utterance comprises generating a spectrogram representation of the user utterance based on a Fourier transform of the recording of the user utterance.