Methods and systems for training a machine learning model and authenticating a user with the model
A method of training a machine learning model (MLM) to conduct authentication transactions includes obtaining, by an electronic device, a training dataset of audio signals. Each audio signal includes voice biometric data of a user and an audio data production flag and belongs to a same or different data class. Moreover, the method includes creating, using a MLM being trained, at least one embedding for each audio signal and calculating, using the at least one embedding, a loss for a machine learning algorithm used for training the MLM. Furthermore, the method includes updating parameters of the MLM based on the calculated loss. In response to determining criteria defining an end of training have been satisfied, deeming the MLM to be operable for use in simultaneously successfully verifying the identity of a user based on voice biometric data and verifying the voice biometric data was spoken by a living person.
1. A method of training a machine learning model to conduct authentication transactions comprising the steps of:
obtaining, by an electronic device, a training dataset of audio signals, wherein each audio signal includes voice biometric data of a user and an audio data production flag and belongs to a same or different data class, each data class comprises a user identity and the audio data production flag or a different audio data production flag;
creating a feature matrix for each audio signal according to the class of data to which the respective audio signal belongs;
inputting the matrices into the machine learning model being trained;
calculating at least one embedding for each of the matrices;
calculating, using the at least one embedding, a loss for a machine learning algorithm (MLA) used for training the machine learning model;
updating parameters of the machine learning model being trained based on the calculated loss;
in response to determining criteria defining an end of training have been satisfied, deeming the machine learning model to be operable for use in simultaneously successfully verifying the identity of a user based on voice biometric data and verifying the voice biometric data was spoken by a living person;
and in response to determining criteria defining an end of training have not been satisfied, repeatedly conducting said creating a feature matrix for each audio signal step, inputting step, calculating at least one embedding for each of the matrices step, calculating, using the at least one embedding, a the loss step, creating, calculating and updating steps until the criteria are satisfied.
2. The method according to claim 1 , wherein the electronic device is a sole electronic device.
3. The method according to claim 1 , said calculating a loss step comprising the step of calculating a metric loss, wherein the metric loss is a triplet loss, a prototypical loss, or a contrastive loss.
4. The method according to claim 1 , said calculating a loss step comprising calculating the loss according to the equations
L
s
=
1
n
s
(
n
s
-
1
)
∑
i
∈
N
s
∑
j
∈
N
s
∖
i
max
(
f
(
x
s
i
)
-
f
(
x
s
j
)
2
2
-
f
(
x
s
i
)
-
f
(
x
r
k
)
2
2
+
α
trp
,
0
)
and
L
trp
=
1
n
∑
s
L
s
Where: Lir p is the triplet loss; Ls is the loss calculated separately for each class s; Matrices x i s and x j s belong to the same class, class s, which means the user and the audio data production flag for the matrices are the same;
Matrix x k r belongs to a different class, class r, at an occurrence k;
Functions f(x i s ),f(x j s ),f(x k s ) are embeddings extracted from the input matrices x i s , x j s and x k r , respectively;
N is an ensemble of different classes of data in the training dataset;
n is the number of elements in ensemble N;Ns is the ensemble of occurrences of audio signals for class s; α erns is the number of elements in ensemble s;
Ns is the ensemble of occurrences of audio signals for class s except for occurrence i; and α er p is a margin of error.
5. An electronic device for training a machine learning model to conduct authentication transactions comprising:
a processor; and a memory configured to store data, said electronic device being associated with a network and said memory being in communication with said processor and having instructions stored thereon which, when read and executed by said processor, cause said electronic device to:
obtain a training dataset of audio signals, wherein each audio signal includes voice biometric data of a user and an audio data production flag each data class comprises a user identity and the audio data production flag or a different audio production flag;
create a feature matrix for each audio signal according to the class of data to which the respective audio signal belongs;
input the matrices into the machine learning model being trained;
calculate at least one embedding for each of the matrices;
calculate, using the at least one embedding, a loss for a machine learning algorithm (MLA) used for training the machine learning model;
update parameters of the machine learning model being trained based on the calculated loss;
in response to determining criteria defining an end of training have been satisfied, deem the machine learning model to be operable for use in simultaneously successfully verifying the identity of a user based on voice biometric data and verifying the voice biometric data was spoken by a living person; and
cause said electronic device to repeatedly:
create the at least one embedding;
create a feature matrix for each audio signal according to the class of data to which the respective audio signal belongs;
input the matrices into the machine learning model being trained;
calculate at least one embedding for each of the matrices;
calculate a loss for the MLA;
and update parameters of the machine learning model being trained based on the calculated loss until the criteria are satisfied.
6. The electronic device according to claim 5 , wherein the electronic device is a sole electronic device.
7. The electronic device according to claim 5 , wherein the instructions when read and executed by said processor, cause said electronic device to calculate the loss as a metric loss, wherein the metric loss is a triplet loss, a prototypical loss, or a contrastive loss.
8. The electronic device according to claim 5 , wherein the instructions when read and executed by said processor, cause said electronic device to calculate the loss according to the equations
L
s
=
1
n
s
(
n
s
-
1
)
∑
i
∈
N
s
∑
j
∈
N
s
∖
i
max
(
f
(
x
s
i
)
-
f
(
x
s
j
)
2
2
-
f
(
x
s
i
)
-
f
(
x
r
k
)
2
2
+
α
trp
,
0
)
and
L
trp
=
1
n
∑
s
L
s
Where: Lir p is the triplet loss; Ls is the loss calculated separately for each class s; Matrices xandx belong to the same class, class s, which means the user and the audio data production flag for the matrices are the same; Matrix x k r belongs to a different class, class r, at an occurrence k;Functions f(x),f(x),f(x x k r) are embeddings extracted from the input matrices x,x and x k r , respectively; N is an ensemble of different classes of data in the training dataset; n is the number of elements in ensemble N; Ns is the ensemble of occurrences of audio signals for class s;ns is the number of elements in ensemble s;Ns\i is the ensemble of occurrences of audio signals for class s except for occurrence i; and arp, is a margin of error.
9. A non-transitory computer-readable recording medium in an electronic device for training a machine learning model to conduct authentication transactions, the non-transitory computer-readable recording medium storing instructions which when executed by a hardware processor cause the non-transitory recording medium to perform steps comprising:
obtaining a training dataset of audio signals, wherein each audio signal includes voice biometric data of a user and an audio data production flag and belongs to a same or different data class, each data class comprises a user identity and the audio data production flag or a different audio data production flag;
creating a feature matrix for each audio signal according to the class of data to which the respective audio signal belongs;
inputting the matrices into the machine learning model being trained:
calculating at least one embedding for each of the matrices;
calculating, using the at least one embedding, a loss for a machine learning algorithm (MLA) used for training the machine learning model; updating parameters of the machine learning model being trained based on the calculated loss; and in response to determining criteria defining an end of training have been satisfied, deeming the machine learning model to be operable for use in simultaneously successfully verifying the identity of a user based on voice biometric data and verifying the voice biometric data was spoken by a living person;
and in response to determining criteria defining an end of training have not been satisfied, repeatedly conducting said creating a feature matrix for each audio signal step, inputting step, calculating at least one embedding for each of the matrices step, calculating, using the at least one embedding, a the loss step, creating, calculating and updating steps until the criteria are satisfied.
10. The non-transitory computer-readable recording medium according to claim 9 , wherein the electronic device is a sole electronic device.
11. The non-transitory computer-readable recording medium according to claim 9 , wherein the instructions when read and executed by said processor, cause said non-transitory computer-readable recording medium to perform steps compromising calculating the loss as a metric loss, wherein the metric loss is a triplet loss, a contractive loss, or a prototypical loss.
12. The non-transitory computer-readable recording medium according to claim 9 , wherein the instructions when read and executed by said processor, cause said non-transitory computer-readable recording medium to repeatedly perform said creating a feature matrix, inputting, calculating at least one embedding, calculating the loss, and updating steps until the criteria are satisfied.
13. The non-transitory computer-readable recording medium according to claim 9 , wherein the instructions when read and executed by said processor, cause said non-transitory computer-readable recording medium to perform steps compromising calculating the loss according to the equations
L
s
=
1
n
s
(
n
s
-
1
)
∑
i
∈
N
s
∑
j
∈
N
s
∖
i
max
(
f
(
x
s
i
)
-
f
(
x
s
j
)
2
2
-
f
(
x
s
i
)
-
f
(
x
r
k
)
2
2
+
α
trp
,
0
)
and
L
trp
=
1
n
∑
s
L
s
Where: Lir p is the triplet loss;Ls is the loss calculated separately for each class s; Matrices xandxi belong to the same class, class s, which means the user and the audio data production flag for the matrices are the same; Matrix x k r belongs to a different class, class r, at an occurrence k;Functions f(x),f(xL),f(x) are embeddings extracted from the input matrices xf,x-′ and x k r , respectively; N is an ensemble of different classes of data in the training dataset; n is the number of elements in ensemble N;Ns is the ensemble of occurrences of audio signals for class s;ns is the number of elements in ensemble s;Ns\i is the ensemble of occurrences of audio signals for class s except for occurrence i and an is a margin of error.
14. A method of authenticating a user comprising the steps of:
prompting a user to speak a phrase;
capturing, by an electronic device, audio data of the user while the user speaks the phrase;
creating a matrix from the captured audio data according to the class of data to which the captured audio data belongs, wherein each data class comprises a user identity and the audio data production flag or a different audio data production flag;
extracting content from the matrix to create at least one embedding;
creating a verification template from the embeddings;
comparing the verification template against a corresponding enrolment template of the user; calculating a matching score for the comparison; comparing the matching score against a threshold score; and in response to determining the matching score is at least equal to the threshold score, simultaneously successfully verifying the identity of the user based on the captured audio data and verifying the captured audio data was spoken by a living person;
and in response to determining criteria defining an end of training have not been satisfied, repeatedly conducting said creating a matrix for each audio data step, capturing step, extracting content from the matrix to create at least one embedding step, creating, comparing step and updating steps until the criteria are satisfied.
15. The method according to claim 14 , said prompting step comprising instructing the user to speak the phrase.