System, apparatus, and method for automatically generating negative keystroke examples and training user identification models based on keystroke dynamics
An apparatus adapted to identify a user based on keystroke dynamics of an input by the user, the apparatus adapted to: execute a first training phase of training a keystroke sample generator to generate negative keystroke samples; execute a second training phase of training a user identification model based at least in part on a plurality of negative keystroke samples generated using the keystroke sample generator; and execute a deployment of the user identification model to authenticate an input sample associated with the user using the trained user identification model.
1 . An apparatus adapted to identify a user based on keystroke dynamics of an input by the user, comprising:
a communication interface to one or more networks;
one or more processing devices operatively connected to the computer network interface; and
one or more memory storage devices operatively connected to the one or more processing devices and having stored thereon machine-readable instructions that, when executed, cause the one or more processing devices to:
in a first training phase of training a keystroke sample generator,
receive, via the communication interface, a plurality of first text input samples by a plurality of first users; and
for each received first text input sample:
generate a user identification representation, a noise representation, and a text sequence representation of the received first text input sample;
generate, using the keystroke sample generator, a keystroke sequence sample based on a combination of the generated user identification representation, noise representation, and text sequence representation;
input the generated keystroke sequence sample and an actual keystroke sequence of the received first text input sample to a classifier for a user identification classification on the generated keystroke sequence sample;
input the generated keystroke sequence sample and the actual keystroke sequence of the received first text input sample to a regressor for a text character length regression on the generated keystroke sequence sample; and
train the keystroke sample generator based on the user identification classification of the classifier and the text character length classification of the regressor;
in a second training phase of training a user identification model,
receive, via the communication interface, one or more second text input samples by a second user, said second user being different from the plurality of first users;
generate, using the keystroke sample generator, a plurality of negative keystroke samples based on the one or more second text input samples; and
train the user identification model on a user classification of the second user based on the one or more second text input samples and the generated plurality of negative keystroke samples; and
in a deployment of the user identification model,
receive, via the communication interface, a third text input sample in association with the second user; and
authenticate the third text input sample using the trained user identification model.
2 . The apparatus of claim 1 , wherein the user identification representation and the text sequence representation are generated using respective embedding neural layers.
3 . The apparatus of claim 1 , wherein the user identification representation and the noise representation are generated to conform to a format of the text sequence representation using respective neural layers.
4 . The apparatus of claim 1 , wherein the classifier is a multi-class discriminator embodied by a neural network and the regressor is a regression neural network.
5 . The apparatus of claim 4 , wherein the training of the keystroke sample generator, the user identification classification by the classifier, and the text character length regression by the regressor are based on
L ( G,D,R,x,u,t )= L GAN ( G,D,x,u,t )+λ 1 ·L time ( G,u,t )+λ 2 ·L MSE ( G,R,x,t )
where:
G represents the keystroke sample generator,
D represents the multi-class discriminator,
R represents the regressor,
x represents an array of press and release timestamps for a typed text sequence of the received first text input sample,
t represents the typed text sequence of the received first text input sample,
u represents a one-hot encoded vector representing an ID associated with one of the plurality of first users that typed the received first text input sample or a label indicating that an input sample to the classifier is generated,
λ 1 is a hyperparameter that controls an importance of a temporal consistency loss (denoted as L time ),
λ 2 is a hyperparameter that controls an importance of a mean square error (MSE) loss (denoted as L MSE ) with respect to the text character length of the generated keystroke sequence sample,
L GAN ( G,D,x,u,t )= E x˜p data (x) [−Σ i=1 k u i ·log( D ( x ))]+ E z˜p z (z) [−Σ i=1 m u i ·log( D ( G ( z|u,t )))]
E represents an expected value ((average) of each formula included in square brackets),
p data represents a probability distribution of data,
x is a data point sampled from the distribution p data ,
E x˜p data (x) is the expected value over all data points,
p z represents a noise distribution,
z is a noise vector sampled from the distribution p z ,
E z˜p z (z) is the expected value over all noise vectors,
m represents a number of the plurality of first users,
k represents a number of the plurality of second users,
L time ( G,u,t )= E z˜p z ( z )[Σ i=1 n max(0, y i,0 −y i,1 )+Σ i=1 n−1 max(0, y i,0 −y i+1,0 )]
y=G(z|u,t), therefore y i,0 and y i,1 represent press and release timestamps of an i-th key in the sequence t,
n represents a sequence length, and
L MSE ( G,R,x,t )= E x˜p data (x) [( R ( x )− len ( t )) 2 ]+E z˜p z (z) [( R ( G ( z|u,t ))− len ( t )) 2 ]
len is a function that returns a length of a sequence t.
6 . The apparatus of claim 1 , wherein the plurality of negative keystroke samples are generated by the generator based on one or more of the plurality of first text input samples in association with one or more of the plurality of first users, which are different from the second user.
7 . The apparatus of claim 6 , wherein at least one of the negative keystroke samples is generated based on one of the plurality of first text input samples that comprises a same character sequence as the one or more second text input samples.
8 . The apparatus of claim 1 , wherein the user identification model is a binary classifier that determines whether a keystroke sequence of the third text input sample corresponds to the second user based on the user classification training.
9 . The apparatus of claim 1 , wherein the plurality of first text input samples comprise free text inputs by the plurality of first users.
10 . The apparatus of claim 1 , wherein the one or more second text input samples comprise a fixed text input by the second user.
11 . A method for identifying a user based on keystroke dynamics of an input by the user, comprising:
in a first training phase of training a keystroke sample generator,
receiving, by a processing apparatus via a communication interface, a plurality of first text input samples by a plurality of first users; and
for each received first text input sample:
generating, by the processing apparatus, a user identification representation, a noise representation, and a text sequence representation of the received first text input sample;
generating, by the processing apparatus using the keystroke sample generator, a keystroke sequence sample based on a combination of the generated user identification representation, noise representation, and text sequence representation;
inputting, by the processing apparatus, the generated keystroke sequence sample and an actual keystroke sequence of the received first text input sample to a classifier for a user identification classification on the generated keystroke sequence sample;
inputting, by the processing apparatus, the generated keystroke sequence sample and the actual keystroke sequence of the received first text input sample to a regressor for a text character length regression on the generated keystroke sequence sample; and
training, by the processing apparatus, the keystroke sample generator based on the user identification classification of the classifier and the text character length classification of the regressor;
in a second training phase of training a user identification model,
receiving, by the processing apparatus via the communication interface, one or more second text input samples by a second user, said second user being different from the plurality of first users;
generating, by the processing apparatus using the keystroke sample generator, a plurality of negative keystroke samples based on the one or more second text input samples; and
training, by the processing apparatus, the user identification model on a user classification of the second user based on the one or more second text input samples and the generated plurality of negative keystroke samples; and
in a deployment of the user identification model,
receiving, by the processing apparatus via the communication interface, a third text input sample in association with the second user; and
authenticating, by the processing apparatus, the third text input sample using the trained user identification model.
12 . The method of claim 11 , wherein the user identification representation and the text sequence representation are generated using respective embedding neural layers.
13 . The method of claim 11 , wherein the user identification representation and the noise representation are generated to conform to a format of the text sequence representation using respective neural layers.
14 . The method of claim 11 , wherein the classifier is a multi-class discriminator embodied by a neural network and the regressor is a regression neural network.
15 . The method of claim 14 , wherein the training of the keystroke sample generator, the user identification classification by the classifier, and the text character length regression by the regressor are based on
L ( G,D,R,x,u,t )= L GAN ( G,D,x,u,t )+λ 1 ·L time ( G,u,t )+λ 2 ·L MSE ( G,R,x,t )
where:
G represents the keystroke sample generator,
D represents the multi-class discriminator,
R represents the regressor,
x represents an array of press and release timestamps for a typed text sequence of the received first text input sample,
t represents the typed text sequence of the received first text input sample,
u represents a one-hot encoded vector representing an ID associated with one of the plurality of first users that typed the received first text input sample or a label indicating that an input sample to the classifier is generated,
λ 1 is a hyperparameter that controls an importance of a temporal consistency loss (denoted as L time ),
λ 2 is a hyperparameter that controls an importance of a mean square error (MSE) loss (denoted as L MSE ) with respect to the text character length of the generated keystroke sequence sample,
L GAN ( G,D,x,u,t )= E x˜p data (x) [−Σ i=1 k u i ·log( D ( x ))]+ E z˜p z (z) [−Σ i=1 m u i ·log( D ( G ( z|u,t )))]
E represents an expected value ((average) of each formula included in square brackets),
p data represents a probability distribution of data,
x is a data point sampled from the distribution p data ,
E x˜p data (x) is the expected value over all data points,
p z represents a noise distribution,
z is a noise vector sampled from the distribution p z ,
E z˜p z (z) is the expected value over all noise vectors,
m represents a number of the plurality of first users,
k represents a number of the plurality of second users,
L time ( G,u,t )= E z˜p z (z) [Σ i=1 n max(0, y i,0 −y i,1 )+Σ i=1 n−1 max(0, y i,0 −y i+1,0 )]
y=G(z|u,t), therefore y i,0 and y i,1 represent press and release timestamps of an i-th key in the sequence t,
n represents a sequence length, and
L MSE ( G,R,x,t )= E x˜p data (x) [( R ( x )− len ( t )) 2 ]+E z˜p z (z) [( R ( G ( z|u,t ))− len ( t ) 2 ]
len is a function that returns a length of a sequence t.
16 . The method of claim 11 , wherein the plurality of negative keystroke samples are generated by the generator based on one or more of the plurality of first text input samples in association with one or more of the plurality of first users, which are different from the second user.
17 . The method of claim 16 , wherein at least one of the negative keystroke samples is generated based on one of the plurality of first text input samples that comprises a same character sequence as the one or more second text input samples.
18 . The method of claim 11 , wherein the user identification model is a binary classifier that determines whether a keystroke sequence of the third text input sample corresponds to the second user based on the user classification training.
19 . The method of claim 11 , wherein the plurality of first text input samples comprise free text inputs by the plurality of first users.
20 . The method of claim 11 , wherein the one or more second text input samples comprise a fixed text input by the second user.