Anomalous event prediction using contrastive learning
Various techniques can include systems and methods for using contrastive learning to predict anomalous events in data processing systems. The method can include accessing an unstructured data file and contextual data associated with the unstructured data file. The method can also include generating an event-data input element for the unstructured data file. The event-data input element can include a set of feature vectors. The set of feature vectors can include a first feature vector generated by using a first encoder to process the unstructured file and a second feature vector generated by using a second encoder to process the contextual data. The method can also include generating a classification result of the unstructured data file by using a machine-learning model to process the event-data input element, in which the classification result includes a prediction of whether the particular event corresponds to an anomalous event.
1 . A method comprising:
tuning a first encoder to process unstructured data files by determining one or more first parameters of the first encoder and tuning a second encoder to process contextual data items by determining one or more second parameters of the second encoder at least in part by:
transforming a first training feature vector generated by the first encoder at in a first feature space to generate a first training transformed feature vector that represents a training unstructured data file in a shared feature space, wherein the training unstructured data file identifies a training event;
transforming a second training feature vector generated by the second encoder in a second feature space to generate a second training transformed feature vector that represents a training contextual data in the shared feature space, wherein the training contextual data identifies the same training event as the training unstructured data file;
determining, using a contrastive loss function, a contrastive loss in the shared feature space between the first training transformed feature vector and the second training transformed feature vector; and
accessing an unstructured data file, wherein the unstructured data file identifies an occurrence of a particular transaction;
accessing contextual data associated with the unstructured data file, wherein the contextual data includes information that describes an environment at which the particular transaction occurred and comprises user profile data of an entity involved in the particular transaction and information from data records reporting the particular transaction; wherein the contextual data is generated by applying one or more pre-processing operations to merge the user profile data and the information from data records reporting the particular transaction;
generating an event-data input element for the unstructured data file, wherein the event-data input element includes a set of feature vectors, and wherein the event-data input element is generated by:
generating a first feature vector of the set of feature vectors by using the first encoder as tuned to process the unstructured data file, wherein the first feature vector represents the unstructured data file in the first feature space;
generating a second feature vector of the set of feature vectors by using the second encoder as tuned to process the contextual data, wherein the second feature vector represents the contextual data in the second feature space;
generating a vector in the shared feature space based on the first feature vector and the second feature vector, wherein the generating comprises combining information from the first feature vector and the second feature vector to result in the vector with improved projected predictiveness over either the first feature vector or the second feature vector in isolation;
generating a classification result of the unstructured data file by using a machine-learning model to process the event-data input element, wherein the classification result includes a prediction of whether the particular transaction corresponds to an anomalous transaction; and
based at least in part on the classification result predicting the particular transaction corresponds to an anomalous transaction, determining to reject the particular transaction.
2 . The method of claim 1 , further comprising:
determining that the first feature vector or the second feature vector deviates from an expected vector value;
in response to determining that the first feature vector or the second feature vector deviates from the expected vector value, receiving feedback data associated with the contextual data, wherein generating the classification result further includes processing the feedback data using the machine-learning model.
3 . The method of claim 1 , wherein the first encoder was trained using a set of training unstructured data files, wherein training of the first encoder includes:
augmenting a training unstructured data file of the set of training unstructured data files to generate a first training augmented unstructured data file and a second training augmented unstructured data file;
generating a third training feature vector of the first training augmented unstructured data file in the first feature space;
generating a fourth training feature vector of the second training augmented unstructured data file in the first feature space;
determining a loss between the third training feature vector and the fourth training feature vector; and
adjusting one or more parameters of the first encoder based on the determined loss.
4 . The method of claim 1 , wherein the second encoder was trained using a set of training contextual-data objects, wherein training of the second encoder includes:
augmenting a training contextual-data object of the set of training contextual-data objects to generate a first training augmented contextual-data object and a second training augmented contextual-data object;
generating a fifth training feature vector of the first training augmented contextual-data object in the second feature space;
generating a sixth training feature vector of the second training augmented contextual-data object in the second feature space;
determining a loss between the fifth training feature vector and the sixth training feature vector; and
adjusting one or more parameters of the second encoder based on the determined loss.
5 . The method of claim 1 , wherein the first encoder includes a convolutional neural network.
6 . The method of claim 1 , wherein the machine-learning model includes a tree-based classifier.
7 . The method of claim 1 , wherein the contextual data includes a first set of data records accessed from a first user profile data store and a second set of data records accessed from a second transaction report data store.
8 . The method of claim 1 , wherein the classification result further includes a severity score of the particular transaction, wherein the severity score is predictive of whether the particular transaction corresponds to a particular type of the anomalous transaction.
9 . The method of claim 1 , further comprising extracting text from the unstructured data file using optical character recognition, and determining overlapping data between the first feature vector and the second feature vector based at least in part on the extracted text.
10 . A non-transitory machine-readable storage medium storing instructions configured to cause one or more data processors to perform operations including:
tuning a first encoder to process unstructured data files by determining one or more first parameters of the first encoder and tuning a second encoder to process contextual data items by determining one or more second parameters of the second encoder at least in part by:
transforming a first training feature vector generated by the first encoder in a first feature space to generate a first training transformed feature vector that represents a training unstructured data file in a shared feature space, wherein the training unstructured data file identifies a training event;
transforming a second training feature vector generated by the second encoder in a second feature space to generate a second training transformed feature vector that represents a training contextual data in the shared feature space, wherein the training contextual data identifies the same training event as the training unstructured data file;
determining, using a contrastive loss function, a contrastive loss in the shared feature space between the first training transformed feature vector and the second training transformed feature vector; and
accessing an unstructured data file, wherein the unstructured data file identifies an occurrence of a particular transaction;
accessing contextual data associated with the unstructured data file, wherein the contextual data includes information that describes an environment at which the particular transaction occurred and comprises user profile data of an entity involved in the particular transaction and information from data records reporting the particular transaction; wherein the contextual data is generated by applying one or more pre-processing operations to merge the user profile data and the information from data records reporting the particular transaction;
generating an event-data input element for the unstructured data file, wherein the event-data input element includes a set of feature vectors, and wherein the event-data input element is generated by:
generating a first feature vector of the set of feature vectors by using the first encoder as tuned to process the unstructured data file, wherein the first feature vector represents the unstructured data file in the first feature space;
generating a second feature vector of the set of feature vectors by using the second encoder as tuned to process the contextual data, wherein the second feature vector represents the contextual data in the second feature space;
generating a vector in the shared feature space based on the first feature vector and the second feature vector, wherein the generating comprises combining information from the first feature vector and the second feature vector to result in the vector with improved projected predictiveness over either the first feature vector or the second feature vector in isolation;
generating a classification result of the unstructured data file by using a machine-learning model to process the event-data input element, wherein the classification result includes a prediction of whether the particular transaction corresponds to an anomalous transaction; and
based at least in part on the classification result predicting the particular transaction corresponds to an anomalous transaction, determining to reject the particular transaction.
11 . The non-transitory machine-readable storage medium of claim 10 , wherein the instructions further cause one or more data processors to perform operations including:
determining that the first feature vector or the second feature vector deviates from an expected vector value;
in response to determining that the first feature vector or the second feature vector deviates from the expected vector value, receiving feedback data associated with the contextual data, wherein generating the classification result further includes processing the feedback data using the machine-learning model.
12 . The non-transitory machine-readable storage medium of claim 10 , wherein the first encoder was trained using a set of training unstructured data files, wherein training of the first encoder includes:
augmenting a training unstructured data file of the set of training unstructured data files to generate a first training augmented unstructured data file and a second training augmented unstructured data file;
generating a third training feature vector of the first training augmented unstructured data file in the first feature space;
generating a fourth training feature vector of the second training augmented unstructured data file in the first feature space;
determining a loss between the third training feature vector and the fourth training feature vector; and
adjusting one or more parameters of the first encoder based on the determined loss.
13 . The non-transitory machine-readable storage medium of claim 10 , wherein the second encoder was trained using a set of training contextual-data objects, wherein training of the second encoder includes:
augmenting a training contextual-data object of the set of training contextual-data objects to generate a first training augmented contextual-data object and a second training augmented contextual-data object;
generating a fifth training feature vector of the first training augmented contextual-data object in the second feature space;
generating a sixth training feature vector of the second training augmented contextual-data object in the second feature space;
determining a loss between the fifth training feature vector and the sixth training feature vector; and
adjusting one or more parameters of the second encoder based on the determined loss.
14 . The non-transitory machine-readable storage medium of claim 10 , wherein the first encoder includes a convolutional neural network.
15 . The non-transitory machine-readable storage medium of claim 10 , wherein the classification result further includes a severity score of the particular transaction, wherein the severity score is predictive of whether the particular event corresponds to a particular type of the anomalous transaction.
16 . The non-transitory machine-readable storage medium of claim 10 , wherein the instructions further cause one or more data processors to perform operations including extracting text from the unstructured data file using optical character recognition, and determining overlapping data between the first feature vector and the second feature vector based at least in part on the extracted text.
17 . The non-transitory machine-readable storage medium of claim 10 , wherein the contextual data includes a first set of data records accessed from a first user profile data store and a second set of data records accessed from a second transaction report data store.
18 . A system comprising:
one or more data processors; and
a non-transitory computer readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform operations including:
tuning a first encoder to process unstructured data files by determining one or more first parameters of the first encoder and tuning a second encoder to process contextual data items by determining one or more second parameters of the second encoder at least in part by:
transforming a first training feature vector generated by the first encoder in a first feature space to generate a first training transformed feature vector that represents a training unstructured data file in a shared feature space, wherein the training unstructured data file identifies a training event;
transforming a second training feature vector generated by the second encoder in a second feature space to generate a second training transformed feature vector that represents a training contextual data in the shared feature space, wherein the training contextual data identifies the same training event as the training unstructured data file;
determining, using a contrastive loss function, a contrastive loss in the shared feature space between the first training transformed feature vector and the second training transformed feature vector; and
accessing an unstructured data file, wherein the unstructured data file identifies an occurrence of a particular transaction;
accessing contextual data associated with the unstructured data file, wherein the contextual data includes information that describes an environment at which the particular transaction occurred and comprises user profile data of an entity involved in the particular transaction and information from data records reporting the particular transaction; wherein the contextual data is generated by applying one or more pre-processing operations to merge the user profile data and the information from data records reporting the particular transaction;
generating an event-data input element for the unstructured data file, wherein the event-data input element includes a set of feature vectors, and wherein the event-data input element is generated by:
generating a first feature vector of the set of feature vectors by using the first encoder as tuned to process the unstructured data file, wherein the first feature vector represents the unstructured data file in the first feature space;
generating a second feature vector of the set of feature vectors by using the second encoder as tuned to process the contextual data, wherein the second feature vector represents the contextual data in the second feature space;
generating a vector in the shared feature space based on the first feature vector and the second feature vector, wherein the generating comprises combining information from the first feature vector and the second feature vector to result in the vector with improved projected predictiveness over either the first feature vector or the second feature vector in isolation;
generating a classification result of the unstructured data file by using a machine-learning model to process the event-data input element, wherein the classification result includes a prediction of whether the particular transaction corresponds to an anomalous transaction; and
based at least in part on the classification result predicting the particular transaction corresponds to an anomalous transaction, determining to reject the particular transaction.
19 . The system of claim 18 , wherein the instructions further cause one or more data processors to perform operations including:
determining that the first feature vector or the second feature vector deviates from an expected vector value;
in response to determining that the first feature vector or the second feature vector deviates from the expected vector value, receiving feedback data associated with the contextual data, wherein generating the classification result further includes processing the feedback data using the machine-learning model.
20 . The system of claim 18 , wherein the first encoder was trained using a set of training unstructured data files, wherein training of the first encoder includes:
augmenting a training unstructured data file of the set of training unstructured data files to generate a first training augmented unstructured data file and a second training augmented unstructured data file;
generating a third training feature vector of the first training augmented unstructured data file in the first feature space;
generating a fourth training feature vector of the second training augmented unstructured data file in the first feature space;
determining a loss between the third training feature vector and the fourth training feature vector; and
adjusting one or more parameters of the first encoder based on the determined loss.
21 . The system of claim 18 , wherein the second encoder was trained using a set of training contextual-data objects, wherein training of the second encoder includes:
augmenting a training contextual-data object of the set of training contextual-data objects to generate a first training augmented contextual-data object and a second training augmented contextual-data object;
generating a fifth training feature vector of the first training augmented contextual-data object in the second feature space;
generating a sixth training feature vector of the second training augmented contextual-data object in the second feature space;
determining a loss between the fifth training feature vector and the sixth training feature vector; and
adjusting one or more parameters of the second encoder based on the determined loss.
22 . The system of claim 18 , wherein the first encoder includes a convolutional neural network.
23 . The system of claim 18 , wherein the classification result further includes a severity score of the particular transaction, wherein the severity score is predictive of whether the particular transaction corresponds to a particular type of the anomalous transaction.
24 . The system of claim 18 , wherein the instructions further cause one or more data processors to perform operations including extracting text from the unstructured data file using optical character recognition, and determining overlapping data between the first feature vector and the second feature vector based at least in part on the extracted text.
25 . The system of claim 18 , wherein the contextual data includes a first set of data records accessed from a first user profile data store and a second set of data records accessed from a second transaction report data store.