Systems and methods for expert guided semi-supervision with contrastive loss for machine learning models
A method includes, in response to at least one convergence criterion not being met: receiving a labeled dataset that includes a plurality of labeled samples; receiving an unlabeled dataset that includes a plurality of unlabeled samples; identifying a plurality of labeled-unlabeled sample pairs; applying a data augmentation transformation to each labeled sample and each corresponding unlabeled sample; computing, for each least one labeled-unlabeled sample pair, latent representation spaces using the machine learning model; generating, using the machine learning model, a label prediction for each unlabeled sample for each labeled-unlabeled sample pair; computing a loss function for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs based on respective latency representation spaces and respective label predictions; applying an optimization function to each respective loss function; and updating a weight value for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs responsive to applying the optimization function.
1 . A method for semi-supervised training of a machine learning model, the method comprising:
determining whether at least one convergence criterion is met; and
in response to the at least one convergence criterion not being met:
receiving a labeled dataset that includes a plurality of labeled samples;
receiving an unlabeled dataset that includes a plurality of unlabeled samples;
identifying a plurality of labeled-unlabeled sample pairs, each labeled-unlabeled sample pair including a respective labeled sample of the labeled samples and a corresponding unlabeled sample of the plurality of unlabeled samples;
applying a data augmentation transformation to each labeled sample and each corresponding unlabeled sample for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs;
computing, for each least one labeled-unlabeled sample pair, latent representation spaces using the machine learning model;
generating, using the machine learning model, a label prediction for each unlabeled sample for each labeled-unlabeled sample pair;
computing a loss function for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs based on respective latency representation spaces and respective label predictions, wherein the loss function includes a contrastive loss term that utilizes a cosine similarity metric between the latent representations;
applying a stochastic gradient descent optimization function to minimize each respective loss function, including the contrastive loss term; and
updating a weight value for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs responsive to applying the optimization function.
2 . The method of claim 1 , wherein the machine learning model includes a feature extractor and one or more predictor networks.
3 . The method of claim 1 , further comprising training the machine learning model using the labeled dataset and the unlabeled dataset having samples propagated with predicted labels.
4 . The method of claim 1 , wherein identifying the plurality of labeled-unlabeled sample pairs includes identifying the plurality of labeled-unlabeled sample pairs using a similarity graph associated with the labeled dataset and the unlabeled dataset.
5 . The method of claim 4 , wherein the similarity graph is generated based on at least an expert derived similarity graph.
6 . The method of claim 1 , wherein the loss function includes a a mean squared error of each label prediction.
7 . The method of claim 1 , wherein the machine learning model is configured to perform at least one classification task.
8 . The method of claim 1 , wherein the machine learning model is configured to perform at least one regression task.
9 . A system for semi-supervised training of a machine learning model, the system comprising:
a processor; and
a memory including instructions that, when executed by the processor, cause the processor to:
determine whether at least one convergence criterion is met; and
in response to the at least one convergence criterion not being met:
receive a labeled dataset that includes a plurality of labeled samples;
receive an unlabeled dataset that includes a plurality of unlabeled samples;
identify a plurality of labeled-unlabeled sample pairs, each labeled-unlabeled sample pair including a respective labeled sample of the labeled samples and a corresponding unlabeled sample of the plurality of unlabeled samples;
apply a data augmentation transformation to each labeled sample and each corresponding unlabeled sample for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs;
compute, for each least one labeled-unlabeled sample pair, latent representation spaces using the machine learning model;
generate, using the machine learning model, a label prediction for each unlabeled sample for each labeled-unlabeled sample pair;
compute a loss function for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs based on respective latency representation spaces and respective label predictions, wherein the loss function includes a contrastive loss term that utilizes a cosine similarity metric between the latent representations;
apply a stochastic gradient descent optimization function to minimize each respective loss function, including the contrastive loss term; and
update a weight value for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs responsive to applying the optimization function.
10 . The system of claim 9 , wherein the machine learning model a feature extractor and one or more predictor networks.
11 . The system of claim 9 , wherein the instructions further cause the processor to train the machine learning model using the labeled dataset and the unlabeled dataset having samples propagated with predicted labels.
12 . The system of claim 9 , wherein the instructions further cause the processor to identify the plurality of labeled-unlabeled sample pairs by identifying the plurality of labeled-unlabeled sample pairs using a similarity graph associated with the labeled dataset and the unlabeled dataset.
13 . The system of claim 12 , wherein the similarity graph is generated based on at least an expert derived similarity graph.
14 . The system of claim 9 , wherein the loss function includes a a mean squared error of each label prediction.
15 . The system of claim 9 , wherein the machine learning model is configured to perform at least one classification task.
16 . The system of claim 9 , wherein the machine learning model is configured to perform at least one regression task.
17 . An apparatus for semi-supervised training of a machine learning model, the apparatus comprising:
a processor; and
a memory including instructions that, when executed by the processor, cause the processor to:
determine whether at least one convergence criterion is met; and
in response to the at least one convergence criterion not being met:
receive a labeled dataset that includes a plurality of labeled samples;
receive an unlabeled dataset that includes a plurality of unlabeled samples;
identify a plurality of labeled-unlabeled sample pairs, each labeled-unlabeled sample pair including a respective labeled sample of the labeled samples and a corresponding unlabeled sample of the plurality of unlabeled samples;
apply a data augmentation transformation to each labeled sample and each corresponding unlabeled sample for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs;
compute, for each least one labeled-unlabeled sample pair, latent representation spaces using a feature extractor of the machine learning model;
generate, using one or more predictor networks of the machine learning model, a label prediction for each unlabeled sample for each labeled-unlabeled sample pair;
compute a loss function for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs based on respective latency representation spaces and respective label predictions, the loss function including a combination of a mean squared error of each label prediction and a contrastive loss that utilizes a cosine similarity metric between the latent representations;
apply a stochastic gradient descent optimization function to minimize each respective loss function, including the contrastive loss term; and
update a weight value for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs responsive to applying the optimization function.
18 . The apparatus of claim 17 , wherein the instructions further cause the processor to train the machine learning model using the labeled dataset and the unlabeled dataset having samples propagated with predicted labels.