Training a machine learning system for transaction data processing
A method of training a supervised machine learning system to detect anomalies within transaction data is described. The method includes obtaining a training set of data samples; assigning a label indicating an absence of an anomaly to unlabelled data samples in the training set; partitioning the data of the data samples in the training set into two feature sets, a first feature set representing observable features and a second feature set representing context features; generating synthetic data samples by combining features from the two feature sets that respectively relate to two different uniquely identifiable entities; assigning a label indicating a presence of an anomaly to the synthetic data samples; augmenting the training set with the synthetic data samples; and training a supervised machine learning system with the augmented training set and the assigned labels.
1 . A method of training a supervised machine learning system to detect anomalies within transaction data stored in a first data storage device using unlabelled data samples of the transaction data, the supervised machine learning system having a binary classifier that outputs a value within a predefined range representing a likelihood of an anomaly, the method comprising:
obtaining, using a payment processor server coupled with the machine learning system, a training set of data samples, each data sample being derived, at least in part, from the transaction data and being associated with one of a set of uniquely identifiable entities, the set of uniquely identifiable entities comprising unique identifiers stored in a second data storage device and relating to a user or a merchant, wherein at least a portion of the training set is unlabelled in which data of said portion of the training set comprises feature vectors that do not have an assigned anomaly label and are thus not indicated as being associated with either normal behaviour or anomalous behaviour;
assigning, using the payment processor server, a label indicating an absence of an anomaly to the unlabelled data samples in the training set and associating the unlabelled data samples with normal behaviour of data that is expected during transaction processing;
partitioning, using the payment processor server, the data of the data samples in the training set into two feature sets, a first feature set representing observable features and a second feature set representing context features, the observable features being derived from a function of at least transaction data for a current transaction, the context features being derived from one or more of a function of historical transaction data that excludes the current transaction and retrieved data relating to the uniquely identifiable entity for the current transaction;
generating, using the payment processor server, synthetic data samples by combining features from the two feature sets that respectively relate to two different entities of the set of uniquely identifiable entities;
assigning, using the payment processor server, a label indicating a presence of an anomaly to the synthetic data samples;
augmenting, using the payment processor server, the training set with the synthetic data samples;
training, using the payment processor server, a supervised machine learning system with the augmented training set and the assigned labels to determine a set of parameters for the supervised machine learning system,
wherein the trained supervised machine learning system is configured to output the value indicative of a presence of an anomaly when supplied with a new data sample by the payment processor server using the set of parameters, thereby training the binary classifier using the unlabelled data samples.
2 . The method of claim 1 , wherein the observable features are derived from a function of transaction data within a predefined temporal window for the current transaction and wherein the context features are derived from transaction data outside of the predefined temporal window.
3 . The method of claim 2 , wherein at least one of the observable features comprise comprises aggregate metrics computed from transaction data for a predefined time period that is defined in relation to a time of the current transaction.
4 . The method of claim 3 , wherein the step of obtaining, using the payment processor server coupled with the machine learning system, with the at least one server the training set of data samples comprises, for a given data sample:
obtaining transaction data for the current transaction, the transaction data comprising an identifier for a uniquely identifiable entity; and
obtaining look-up data for the uniquely identifiable entity;
wherein the obtained transaction data is used to derive the first feature set and the obtained look-up data is used to derive the second feature set.
5 . The method of claim 4 , wherein the step of obtaining, using the payment processor server coupled with the machine learning system, with the at least one server the training set of data samples further comprises, for the given data sample:
obtaining transaction data for the uniquely identifiable entity for the predefined temporal window; and
computing at least one aggregated metric from the transaction data, wherein the at least one aggregated metric is used to derive the first feature set.
6 . The method of claim 4 , wherein the step of obtaining, using the payment processor server coupled with the machine learning system, with the at least one server the training set of data samples further comprises, for the given data sample:
obtaining the historical transaction data for the uniquely identifiable entity, the historical transaction data comprising transaction data that is outside the predefined temporal window; and
computing at least one aggregated metric from the historical transaction data, wherein at least one aggregated metric is used to derive the second feature set.
7 . The method of claim 1 , wherein the second feature set comprises metadata associated with a corresponding uniquely identifiable entity.
8 . The method of claim 1 , wherein the supervised machine learning system comprises a binary classifier that outputs a value within a predefined range representing a likelihood of an anomaly and wherein the labels indicating the absence or presence of an anomaly comprise two numeric values representing a binary output.
9 . The method of claim 1 , wherein the supervised machine learning system comprises an ensemble system based on a set of decision trees.
10 . The method of claim 1 , wherein the supervised machine learning system comprises a recurrent neural network.