MACHINE-LEARNING BASED SECURITY EVENT DETECTION
Systems and methods provide for security event detection using machine learning. Event data items are processed using a first machine-learning model to generate an encoding for each corresponding event data item. Each encoding is processed using a second machine learning model to generate a classification indicating whether the corresponding event is fraudulent. A security event is determined based on some of the generated classifications. In response to detecting the security event, an event processing rate is adjusted.
1 . A computer-implemented method comprising:
obtaining a plurality of event data items corresponding to a plurality of events;
processing each event data item of the plurality of event data items using a first machine learning (ML) model to generate an encoding for each corresponding event data item;
detecting a security event based at least in part on the generated encodings, wherein the detecting comprises:
processing each encoding for each event data item using a second ML model to generate a classification of whether the event is fraudulent; and
determining the security event based at least in part on at least some of the classifications; and
in response to detecting the security event, adjusting an event processing rate.
2 . The computer-implemented method of claim 1 , wherein each event data item comprises a first set of features associated with the corresponding event.
3 . The computer-implemented method of claim 2 , wherein generating the encoding for each corresponding event data item comprises:
processing the first set of features using the first ML model to generate a second set of predicted features, wherein the first set of features and the second set of predicted features are mutually exclusive; and
processing the first set of features and the second set of predicted features using the first ML model to generate the encoding.
4 . The computer-implemented method of claim 1 , wherein each event is associated with a timestamp, and wherein the classification of a subsequent event is based on the encoding generated for a prior event of the plurality of events based on timestamps associated with the corresponding event.
5 . The computer-implemented method of claim 1 , wherein determining the security event comprises determining a percentage of some of the events that were classified as fraudulent.
6 . The computer-implemented method of claim 5 , wherein determining the security event comprises determining whether the percentage of the events that were classified as fraudulent is more than a threshold limit.
7 . The computer-implemented method of claim 1 , wherein the first ML model is a generative model and wherein training the first ML model comprises:
preparing an event dataset comprising of a plurality of training samples, wherein each training sample comprises a first set of training features and a second set of training features, and wherein each training sample is associated with a training timestamp and a training label indicating whether the training sample corresponds to a fraudulent event;
processing the first set of training features using the first ML model to generate a second set of predicted training features;
processing the first set of training features and the second set of predicted training features using the first ML model to generate a first encoding;
processing the first set of training features and the second set of training features using the first ML model to generate a second encoding; and
adjusting a plurality of parameters of the first ML model based on the first encoding and the second encoding.
8 . The computer-implemented method of claim 7 , wherein the second ML is a classification model and wherein training the second ML model comprises:
processing the first set of training features and the second set of predicted training features using the first ML model to generate a third encoding;
processing the third encoding using a second ML model to generate a predicted classification indicating whether the training sample corresponds to a fraudulent event; and
adjusting a plurality of parameters of the second ML model based on the predicted classification and the training label associated with the training sample.
9 . The computer-implemented method of claim 1 , wherein detecting the security event comprises:
processing each encoding for each event data item using a second ML model to generate a respective vector representation;
selecting a latent vector for each vector representation based on a distance between the respective vector representations and the selected latent vector;
determining an entropy for each of the selected latent vectors; and
determining the security event based at least in part on the entropy determined for at least one of the selected latent vectors.
10 . A computer system, comprising:
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors to perform operations comprising:
obtaining a plurality of event data items corresponding to a plurality of events;
processing each event data item of the plurality of event data items using a first machine learning (ML) model to generate an encoding for each corresponding event data item;
detecting a security event based at least in part on the generated encodings, wherein the detecting comprises:
processing each encoding for each event data item using a second ML model to generate a classification of whether the event is fraudulent, and
determining the security event based at least in part on at least some of the classifications; and
in response to detecting the security event, adjusting an event processing rate.
11 . The computer system of claim 10 , wherein each event data item comprises a first set of features associated with the corresponding event.
12 . The computer system of claim 11 , wherein generating the encoding for each corresponding event data item comprises:
processing the first set of features using the first ML model to generate a second set of predicted features, wherein the first set of features and the second set of predicted features are mutually exclusive; and
processing the first set of features and the second set of predicted features using the first ML model to generate the encoding.
13 . The computer system of claim 10 , wherein each event is associated with a timestamp, and wherein the classification of a subsequent event is based on the encoding generated for a prior event of the plurality of events based on timestamps associated with the corresponding event.
14 . The computer system of claim 10 , wherein determining the security event comprises determining a percentage of some of the events that were classified as fraudulent.
15 . The computer system of claim 10 , wherein the first ML model is a generative model and wherein training the first ML model comprises:
preparing an event dataset comprising of a plurality of training samples, wherein each training sample comprises a first set of training features and a second set of training features, and wherein each training sample is associated with a training timestamp and a training label indicating whether the training sample corresponds to a fraudulent event;
processing the first set of training features using the first ML model to generate a second set of predicted training features;
processing the first set of training features and the second set of predicted training features using the first ML model to generate a first encoding;
processing the first set of training features and the second set of training features using the first ML model to generate a second encoding; and
adjusting a plurality of parameters of the first ML model based on the first encoding and the second encoding.
16 . The computer system of claim 10 , wherein detecting the security event comprises:
processing each encoding for each event data item using a second ML model to generate a respective vector representation;
selecting a latent vector for each vector representation based on a distance between the respective vector representations and the selected latent vector;
determining an entropy for each of the selected latent vectors; and
determining the security event based at least in part on the entropy determined for at least one of the selected latent vectors.
17 . A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system, the one or more programs including instructions for:
obtaining a plurality of event data items corresponding to a plurality of events;
processing each event data item of the plurality of event data items using a first machine learning (ML) model to generate an encoding for each corresponding event data item;
detecting a security event based at least in part on the generated encodings, wherein the detecting comprises:
processing each encoding for each event data item using a second ML model to generate a classification of whether the event is fraudulent, and
determining the security event based at least in part on at least some of the classifications; and
in response to detecting the security event, adjusting an event processing rate.
18 . The non-transitory computer-readable storage medium of claim 17 , wherein each event is associated with a timestamp, and wherein the classification of a subsequent event is based on the encoding generated for a prior event of the plurality of events based on timestamps associated with the corresponding event.
19 . The non-transitory computer-readable storage medium of claim 17 , wherein the first ML model is a generative model and wherein training the first ML model comprises:
preparing an event dataset comprising of a plurality of training samples, wherein each training sample comprises a first set of training features and a second set of training features, and wherein each training sample is associated with a training timestamp and a training label indicating whether the training sample corresponds to a fraudulent event;
processing the first set of training features using the first ML model to generate a second set of predicted training features;
processing the first set of training features and the second set of predicted training features using the first ML model to generate a first encoding;
processing the first set of training features and the second set of training features using the first ML model to generate a second encoding; and
adjusting a plurality of parameters of the first ML model based on the first encoding and the second encoding.
20 . The non-transitory computer-readable storage medium of claim 19 , wherein the second ML is a classification model and wherein training the second ML model comprises:
processing the first set of training features and the second set of predicted training features using the first ML model to generate a third encoding;
processing the third encoding using a second ML model to generate a predicted classification indicating whether the training sample corresponds to a fraudulent event; and
adjusting a plurality of parameters of the second ML model based on the predicted classification and the training label associated with the training sample.