Generalized expectation maximization for semi-supervised learning
Techniques are described that extend supervised machine-learning algorithms for use with semi-supervised training. Random labels are assigned to unlabeled training data, and the data is split into k partitions. During a label-training iteration, each of these k partitions is combined with the labeled training data, and the combination is used train a single instance of the machine-learning model. Each of these trained models are then used to predict labels for data points in the k−1 partitions of previously-unlabeled training data that were not used to train of the model. Thus, every data point in the previously-unlabeled training data obtains k−1 predicted labels. For each data point, these labels are aggregated to obtain a composite label prediction for the data point. After the labels are determined via one or more label-training iterations, a machine-learning model is trained on data with the resulting composite label predictions and on the labeled data set.
1. A computer-executed method comprising:
assigning a random label to each data point in an unlabeled set of data to produce a working set of data;
partitioning the working set of data into a plurality of data partitions;
producing a respective set of predicted labels corresponding to each data point in the working set of data by, for each given data partition of the plurality of data partitions:
training an interim machine-learning model based on a set of labeled data and the given data partition to produce a trained interim machine-learning model instance corresponding to the given data partition, and
applying the trained interim machine-learning model instance to predict a predicted label for data points in a set of two or more data partitions, of the plurality of data partitions, other than the given data partition,
wherein each predicted label in the respective set of predicted labels of a particular data point in the working set of data is predicted by a respective interim machine-learning model instance trained based on a corresponding data partition, of the plurality of data partitions, other than a particular data partition containing the particular data point;
for each given data point of the working set of data:
based on the set of predicted labels corresponding to the given data point, generating a composite predicted label, and
applying the composite predicted label to the given data point in the working set of data;
after applying the composite predicted label to each data point, of the working set of data, training a particular machine-learning model based, at least in part, on the working set of data and the set of labeled data; and
applying the trained particular machine-learning model to infer a prediction for one or more inference data points;
wherein the method is performed by one or more computing devices.
2. The computer-executed method of claim 1 , further comprising, prior to training the particular machine-learning model and after applying the composite predicted label to each data point, of the working set of data, using the working set of data to produce a respective second set of predicted labels corresponding to each data point in the working set of data by, for each given data partition of the plurality of data partitions:
training a second interim machine-learning model based on the set of labeled data and the given data partition to produce a second trained interim machine-learning model instance corresponding to the given data partition, and
applying the second trained interim machine-learning model instance to predict a second predicted label for data points in a set of two or more data partitions, of the plurality of data partitions, other than the given data partition to generate a second set of predicted labels; and
for each given data point of the working set of data:
based on the second set of predicted labels corresponding to the given data point, generating a second composite predicted label, and
applying the second composite predicted label to the given data point in the working set of data,
wherein training the particular machine-learning model is performed after applying the second composite predicted label to each data point of the working set of data.
3. The computer-executed method of claim 1 , wherein assigning a random label to each data point in the unlabeled set of data to produce the working set of data comprises:
identifying a particular label, of a plurality of labels, for said each data point based, at least in part, on a plurality of percentages;
wherein each label, of the plurality of labels, is associated with a respective percentage of the plurality of percentages.
4. The computer-executed method of claim 1 , wherein:
the particular machine-learning model is a classification model; and
generating a particular composite predicted label for a particular data point, of the working set of data, based on a particular set of predicted labels corresponding to the particular data point comprises determining the particular composite predicted label based on a mode of the particular set of predicted labels.
5. The computer-executed method of claim 1 , wherein:
the particular machine-learning model is a regression model; and
generating a particular composite predicted label for a particular data point, of the working set of data, based on a particular set of predicted labels corresponding to the particular data point comprises determining the particular composite predicted label based on one or more of:
a mode of the particular set of predicted labels,
a median of the particular set of predicted labels,
a geometric mean of the particular set of predicted labels,
a mean of the particular set of predicted labels, or
a standard deviation of the particular set of predicted labels.
6. The computer-executed method of claim 1 , wherein applying the trained particular machine-learning model to infer the prediction for the one or more inference data points comprises inferring whether the one or more inference data points reflect an anomaly.
7. The computer-executed method of claim 1 , wherein the particular machine-learning model is one of: a Random Forest model, or an artificial neural network.
8. One or more non-transitory computer-readable media storing one or more sequences of instructions that, when executed by one or more processors, cause:
assigning a random label to each data point in an unlabeled set of data to produce a working set of data;
partitioning the working set of data into a plurality of data partitions;
producing a respective set of predicted labels corresponding to each data point in the working set of data by, for each given data partition of the plurality of data partitions:
training an interim machine-learning model based on a set of labeled data and the given data partition to produce a trained interim machine-learning model instance corresponding to the given data partition, and
applying the trained interim machine-learning model instance to predict a predicted label for data points in a set of two or more data partitions, of the plurality of data partitions, other than the given data partition,
wherein each predicted label in the respective set of predicted labels of a particular data point in the working set of data is predicted by a respective interim machine-learning model instance trained based on a corresponding data partition, of the plurality of data partitions, other than a particular data partition containing the particular data point;
for each given data point of the working set of data:
based on the set of predicted labels corresponding to the given data point, generating a composite predicted label, and
applying the composite predicted label to the given data point in the working set of data;
after applying the composite predicted label to each data point, of the working set of data, training a particular machine-learning model based, at least in part, on the working set of data and the set of labeled data; and
applying the trained particular machine-learning model to infer a prediction for one or more inference data points.
9. The one or more non-transitory computer-readable media of claim 8 , wherein the one or more sequences of instructions further comprise instructions that, when executed by one or more processors, cause, prior to training the particular machine-learning model and after applying the composite predicted label to each data point, of the working set of data, using the working set of data to produce a respective second set of predicted labels corresponding to each data point in the working set of data by, for each given data partition of the plurality of data partitions:
training a second interim machine-learning model based on the set of labeled data and the given data partition to produce a second trained interim machine-learning model instance corresponding to the given data partition, and
applying the second trained interim machine-learning model instance to predict a second predicted label for data points in a set of data partitions, of the plurality of data partitions, other than the given data partition to generate a second set of predicted labels; and
for each given data point of the working set of data:
based on the second set of predicted labels corresponding to the given data point, generating a second composite predicted label, and
applying the second composite predicted label to the given data point in the working set of data,
wherein training the particular machine-learning model is performed after applying the second composite predicted label to each data point of the working set of data.
10. The one or more non-transitory computer-readable media of claim 8 , wherein assigning a random label to each data point in the unlabeled set of data to produce the working set of data comprises:
identifying a particular label, of a plurality of labels, for said each data point based, at least in part, on a plurality of percentages;
wherein each label, of the plurality of labels, is associated with a respective percentage of the plurality of percentages.
11. The one or more non-transitory computer-readable media of claim 8 , wherein:
the particular machine-learning model is a classification model; and
generating a particular composite predicted label for a particular data point, of the working set of data, based on a particular set of predicted labels corresponding to the particular data point comprises determining the particular composite predicted label based on a mode of the particular set of predicted labels.
12. The one or more non-transitory computer-readable media of claim 8 , wherein:
the particular machine-learning model is a regression model; and
generating a particular composite predicted label for a particular data point, of the working set of data, based on a particular set of predicted labels corresponding to the particular data point comprises determining the particular composite predicted label based on one or more of:
a mode of the particular set of predicted labels,
a median of the particular set of predicted labels,
a geometric mean of the particular set of predicted labels,
a mean of the particular set of predicted labels, or
a standard deviation of the particular set of predicted labels.
13. The one or more non-transitory computer-readable media of claim 8 , wherein applying the trained particular machine-learning model to infer the prediction for the one or more inference data points comprises inferring whether the one or more inference data points reflect an anomaly.
14. The one or more non-transitory computer-readable media of claim 8 , wherein the particular machine-learning model is one of: a Random Forest model, or an artificial neural network.