System and method for learning with noisy labels as semi-supervised learning
A method provides learning with noisy labels. The method includes generating a first network of a machine learning model with a first set of parameter initial values, and generating a second network of the machine learning model with a second set of parameter initial values. First clean probabilities for samples in a training dataset are generated using the second network. A first labeled dataset and a first unlabeled dataset are generated from the training dataset based on the first clean probabilities. The first network is trained based on the first labeled dataset and first unlabeled dataset to update parameters of the first network.
1. A method for learning with noisy labels, comprising:
generating a first network of a machine learning model with a first set of parameter initial values;
generating a second network of the machine learning model with a second set of parameter initial values;
generating first clean probabilities for a first plurality of samples in a training dataset using the second network;
generating a first labeled dataset including a second plurality of clean samples from the training dataset, wherein the second plurality of clean samples are determined based on the corresponding first clean probabilities;
generating a first unlabeled dataset including a third plurality of noisy samples from the training dataset, wherein the third plurality of noisy samples are determined based on the corresponding first clean probabilities; and
training the first network based on the first labeled dataset and the first unlabeled dataset to update parameters of the first network.
2. The method of claim 1 , further comprising:
generating second clean probabilities for the first plurality of samples in the training dataset using the first network;
generating a second labeled dataset and a second unlabeled dataset from the training dataset based on the second clean probabilities; and
training the second network based on the second labeled dataset and the second unlabeled dataset to update parameters of the second network.
3. The method of claim 1 , wherein the generating the first clean probabilities includes:
modeling a per-sample loss distribution of the second network using a mixture model.
4. The method of claim 3 , wherein the mixture model includes a Gaussian Mixture Model (GMM).
5. The method of claim 1 , wherein the second plurality of clean samples are generated based on a comparison of the first clean probabilities and a clean probability threshold.
6. The method of claim 1 , wherein the training the first network based on the first labeled dataset and the first unlabeled dataset includes:
performing a mix-match process to generate a first mixed labeled dataset and a first mixed unlabeled dataset based on the first labeled dataset and first unfabled dataset;
generating a first total loss including a first loss on the first mixed labeled dataset and a second loss on the first mixed unlabeled dataset; and
updating the parameters of the first network based on the first total loss.
7. The method of claim 6 , wherein the training the first network based on the first labeled dataset and first unlabeled dataset includes:
performing label co-refinement to labels in the first labeled dataset using the first clean probabilities; and
performing label co-guessing to generate guessed labels for the third plurality of noisy samples in the first unlabeled dataset based on predictions from the first network and the second network.
8. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to perform a method comprising:
generating a first network of a machine learning model with a first set of parameter initial values;
generating a second network of the machine learning model with a second set of parameter initial values;
generating first clean probabilities for a first plurality of samples in a training dataset using the second network;
generating a first labeled dataset including a second plurality of clean samples from the training dataset, wherein the second plurality of clean samples are determined based on the corresponding first clean probabilities;
generating a first unlabeled dataset including a third plurality of noisy samples from the training dataset, wherein the third plurality of noisy samples are determined based on the corresponding first clean probabilities; and
training the first network based on the first labeled dataset and first unlabeled dataset to update parameters of the first network.
9. The non-transitory machine-readable medium of claim 8 , the method further comprising:
generating second clean probabilities for the first plurality of samples in the training dataset using the first network;
generating a second labeled dataset and a second unlabeled dataset from the training dataset based on the second clean probabilities; and
training the second network based on the second labeled dataset and second unlabeled dataset to update parameters of the second network.
10. The non-transitory machine-readable medium of claim 8 , wherein the generating the first clean probabilities includes:
modeling a per-sample loss distribution of the second network using a mixture model.
11. The non-transitory machine-readable medium of claim 10 , wherein the mixture model includes a Gaussian Mixture Model (GMM).
12. The non-transitory machine-readable medium of claim 8 , wherein the first labeled dataset is generated based on a comparison of the first clean probabilities and a clean probability threshold.
13. The non-transitory machine-readable medium of claim 8 , wherein the training the first network based on the first labeled dataset and the first unlabeled dataset includes:
performing a mix-match process to generate a first mixed labeled dataset and a first mixed unlabeled dataset based on the first labeled dataset and first unfabled dataset;
generating a first total loss including a first loss on the first mixed labeled dataset and a second loss on the first mixed unlabeled dataset; and
updating the parameters of the first network based on the first total loss.
14. The non-transitory machine-readable medium of claim 13 , wherein the training the first network based on the first labeled dataset and first unlabeled dataset includes:
performing label co-refinement to labels in the first labeled dataset using the first clean probabilities; and
performing label co-guessing to generate guessed labels for the third plurality of noisy samples in the first unlabeled dataset based on predictions from the first network and the second network.
15. A system, comprising:
a non-transitory memory; and
one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform a method comprising:
generating a first network of a machine learning model with a first set of parameter initial values;
generating a second network of the machine learning model with a second set of parameter initial values;
generating first clean probabilities for a first plurality of samples in a training dataset using the second network;
generating a first labeled dataset including a second plurality of clean samples from the training dataset, wherein the second plurality of clean samples are determined based on the corresponding first clean probabilities;
generating a first unlabeled dataset including a third plurality of noisy samples from the training dataset, wherein the third plurality of noisy samples are determined based on the corresponding first clean probabilities; and
training the first network based on the first labeled dataset and the first unlabeled dataset to update parameters of the first network.
16. The system of claim 15 , the method further comprising:
generating second clean probabilities for the first plurality of samples in the training dataset using the first network;
generating a second labeled dataset and a second unlabeled training dataset from the training dataset based on the second clean probabilities; and
training the second network based on the second labeled dataset and second unlabeled training dataset to update parameters of the second network.
17. The system of claim 15 , wherein the generating first clean probabilities includes:
modeling a per-sample loss distribution of the second network using a mixture model.
18. The system of claim 15 , wherein the first labeled dataset is generated based on a clean probability threshold.
19. The system of claim 15 , wherein the training the first network based on the first labeled dataset and the first unlabeled dataset includes:
performing a mix-match process to generate a first mixed labeled dataset and a first mixed unlabeled dataset based on the first labeled dataset and first unfabled dataset;
generating a first total loss including a first loss on the first mixed labeled dataset and a second loss on the first mixed unlabeled dataset; and
updating the parameters of the first network based on the first total loss.
20. The system of claim 19 , wherein the training the first network based on the first labeled dataset and first unlabeled dataset includes:
performing label co-refinement to labels in the first labeled dataset using the first clean probabilities; and
performing label co-guessing to generate guessed labels for the third plurality of noisy samples in the first unlabeled dataset based on predictions from the first network and the second network.