Systems and methods for correcting label bias in data

1. A system comprising:

a memory having instructions stored thereon, and a processor configured to read the instructions to:

obtain a dataset from a database;

select a first portion of the dataset including trusted data of the dataset such that a remaining dataset exists, wherein the first portion of the dataset is selected using stratified sampling of the dataset;

generate a first classification model based on the remaining dataset;

generate an updated untrusted dataset by simulating exposure of the remaining dataset to an adversarial process;

generate a second classification model based on the updated untrusted dataset;

for each data sample of the remaining dataset, determine whether a corresponding observed label is a true label for the data sample based at least in part on the first portion of the dataset, wherein the determination is based on an estimated reverse transformation function for the first classification model and the second classification model;

generate an updated remaining dataset based on the determination, for each data sample of the remaining dataset, whether the observed label is a true label for the data sample; and

train a machine learning model based on the updated remaining dataset and the first portion of the dataset by:

for each data sample of the remaining dataset, determining a probability of the corresponding observed label being a true label; and

determining, for each data sample of the remaining dataset, whether the corresponding observed label is a true label based on the probability, wherein determining whether the corresponding observed label is the true label is based on a confidence of the processor to have correctly determined the probability.

2. The system of claim 1 , wherein the probability is determined based on one or more of a first probability of the observed label being the true label for the corresponding data sample, a second probability of the true label for the corresponding data sample being a trusted label from the first portion of the dataset given the observed label of the data sample, and a probability distribution of features of the data sample having the observed label and the corresponding trusted label.

3. The system of 1 , wherein the processor is further configured to:

determine a portion of the remaining dataset with a set of first data samples where corresponding observed labels are not true labels; and

update the portion of the remaining dataset by replacing the corresponding labels with a trusted label from the first portion of the dataset.

4. The system of claim 1 , wherein a size of the first portion of the dataset is based at least in part on minimizing a weighted sum of sampling variances of features associated with the dataset.

5. The system of claim 1 , wherein a size of the first portion of the dataset is based at least in part on a linear cost function.

6. The system of claim 1 , wherein for each data sample of the remaining dataset, determining whether a corresponding observed label is a true label is based at least in part on assuming that the true labels of the data samples are related to the corresponding observed labels.

7. The system of claim 1 , wherein the updated remaining dataset is generated as the remaining dataset based on the determination, for each data sample of the remaining dataset, that the corresponding observed label is a true label for the data sample.

8. A computer-implemented method comprising:

obtaining a dataset from a database;

selecting a first portion of the dataset including trusted data of the dataset such that a remaining dataset exists, wherein the first portion of the dataset is selected using stratified sampling of the dataset;

generating a first classification model based on the remaining dataset;

generating an updated untrusted dataset by simulating exposure of the remaining dataset to an adversarial process;

generating a second classification model based on the updated untrusted dataset;

for each data sample of the remaining dataset, determining whether a corresponding observed label is a true label for the data sample based at least in part on the first portion of the dataset, wherein the determination is based on an estimated reverse transformation function for the first classification model and the second classification model;

generating an updated remaining dataset based on the determination, for each data sample of the remaining dataset, whether the observed label is a true label for the data sample and performing at least one operation based at least in part on the updated remaining dataset; and

training a machine learning model based on the updated remaining dataset and the first portion of the dataset by:

for each data sample of the remaining dataset, determining a probability of the corresponding observed label being a true label; and

determining, for each data sample of the remaining dataset, whether the corresponding observed label is a true label based on the probability, wherein determining whether the corresponding observed label is the true label is based on a confidence of having correctly determined the probability.

9. The method of claim 8 , wherein the probability is determined based on one or more of a first probability of the observed label being the true label for the corresponding data sample, a second probability of the true label for the corresponding data sample being a trusted label from the first portion of the dataset given the observed label of the data sample, and a probability distribution of features of the data sample having the observed label and the corresponding trusted label.

10. The method of claim 8 , the method further comprising:

determining a portion of the remaining dataset with a set of first data samples where corresponding observed labels are not true labels; and

updating the portion of the remaining dataset by replacing the corresponding labels with a trusted label from the first portion of the dataset.

11. The method of claim 8 , wherein a size of the first portion of the dataset is based at least in part on minimizing a weighted sum of sampling variances of features associated with the dataset.

12. The method of claim 8 , wherein a size of the first portion of the dataset is based at least in part on a linear cost function.

13. The method of claim 8 , wherein for each data sample of the remaining dataset, determining whether a corresponding observed label is a true label is based at least in part on assuming that the true labels of the data samples are related to the corresponding observed labels.

14. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause a device to perform operations comprising:

obtaining training data;

dividing the training data into a trusted dataset and an untrusted dataset, wherein the trusted dataset is selected using stratified sampling;

generating a first classification model based on the remaining dataset;

generating an updated untrusted dataset by simulating exposure of the untrusted dataset to an adversarial process;

generating a second classification model based on the updated untrusted dataset;

updating labels in the untrusted dataset to generate updated untrusted dataset based at least in part on trusted labels in the trusted dataset and injecting noise in the untrusted dataset, wherein the updating is based on an estimated reverse transformation function for the first classification model and the second classification model;

generating updated training data by combining the trusted dataset and updated untrusted dataset; and

training a machine learning model based on the updated training data by:

for each data sample of the untrusted dataset, determining a probability of the corresponding observed label being a true label; and

determining, for each data sample of the untrusted dataset, whether the corresponding observed label is a true label based on the probability, wherein determining whether the corresponding observed label is the true label is based on a confidence of having correctly determined the probability.