Binary signal classifiers that tolerate incorrect training data

1. A method of automatically tolerating training data that is incorrect when training a machine learning binary classifier, the method comprising:

training the machine learning binary classifier using a set of training data entries, each training data entry having known inputs and a known output label, wherein the set of training data entries includes a subset of training data entries that have incorrectly labeled known output labels for both a binary true state and a binary false state of the output labels;

wherein training comprises:

determining a correct likelihood ratio that each training data entry of the set of training data entries has a correctly labeled output label and an incorrect likelihood ratio that each training data entry of the set of training data entries has an incorrectly labeled output label;

identifying a correct a priori odds ratio that the set of training data entries have correctly labeled output labels and an incorrect a priori odds ratio that the set of training data entries have incorrectly labeled output labels;

calculating a correct probability that each entry of the set of entries has a correctly labeled output label using the correct likelihood ratio for that entry and the correct a priori odds ratio;

calculating an incorrect probability that each entry of the set of entries has an incorrectly labeled output label using the incorrect likelihood ratio for that entry and the incorrect a priori odds ratio; and

training the machine learning binary classifier using a tolerant logistic regression model that combines the correct probability and the incorrect probability, wherein training the machine learning binary classifier using a logistic regression model includes adding the correct probability and the incorrect probability.

2. The method of claim 1 , wherein:

the tolerant logistic regression model includes a first and second logistic regression model;

determining the correct likelihood ratio comprises fitting the first logistic regression model to each entry of the training data and determining the incorrect likelihood ratio comprises fitting the second logistic regression model to each entry of the training data; and

wherein fitting comprises minimizing a likelihood function that each entry of the training data fits a logistic regression model to estimate model parameters of a logistic regression model.

3. The method of claim 2 , wherein:

calculating the correct probability for each entry includes multiplying the correct likelihood ratio and the correct a priori odds ratio, and

calculating the incorrect probability for each entry includes multiplying the incorrect likelihood ratio and the incorrect a priori odds ratio.

4. The method of claim 3 , wherein training the machine learning binary classifier using a logistic regression model includes adding the correct probability and the incorrect probability.

5. The method of claim 1 , wherein the machine learning binary classifier includes one of a logistic regression model, a neural network, a perceptron model, a multi-layer perceptron model, a naive Bayes/Bayes model, a decision tree model, a random forest model or a deep neural network model.

6. The method of claim 1 , wherein identifying the correct a priori odds ratio and the incorrect a priori odds ratio comprises one of a machine detecting or a person observing a percentage of a different set of data entries that have correctly labeled predicted output labels after being estimated by a trained version of the machine learning binary classifier.

7. The method of claim 1 , further comprising:

creating the set of training data entries by using one of a machine detecting or a person observing the known output labels of the set of training data entries based on observing the known inputs of the set of training data entries.

8. The method of claim 2 , wherein:

calculating the correct probability for each entry includes adding the correct likelihood ratio and the correct a priori odds ratio, and

calculating the incorrect probability for each entry includes adding the incorrect likelihood ratio and the incorrect a priori odds ratio.

9. The method of claim 1 , wherein the correct likelihood ratio is a first sigmoid term for a label li being correct, and the incorrect likelihood ratio is a second sigmoid term for the label li being incorrect that is nearly a mirror image of the first sigmoid term.

10. The method of claim 2 , further comprising:

plotting the correct likelihood ratio and incorrect likelihood ratio versus a logit of the known inputs for the set of training entries using the binary classifier trained with the set of training entries; and

determining an approximated correct likelihood ratio and an approximated incorrect likelihood ratio by performing an estimation of the plots.

11. A system for automatically tolerating training data that is incorrect using a machine learning binary classifier, the system comprising:

a tolerant logistic regression model trained with a set of training data entries, each training data entry having known inputs and a known output label, wherein the set of training data entries includes a subset of training data entries that have incorrectly labeled known output labels;

the trained logistic regression model having parameters based on:

a correct likelihood ratio that each training data entry of the set of training data entries has a correctly labeled output label and an incorrect likelihood ratio that each training data entry of the set of training data entries has an incorrectly labeled output label for both a binary true state and a binary false state of the output labels;

a correct a priori odds ratio that the set of training data entries have correctly labeled output labels and an incorrect a priori odds ratio that the set of training data entries have incorrectly labeled output labels;

a correct probability that each entry of the set of entries has a correctly labeled output label that includes the correct likelihood ratio for that entry and the correct a priori odds ratio;

an incorrect probability that each entry of the set of entries has an incorrectly labeled output label that includes the incorrect likelihood ratio for that entry and the incorrect a priori odds ratio; and

a combination of the correct probability and the incorrect probability, wherein the combination of the correct probability and the incorrect probability is an addition of the correct probability and the incorrect probability.

12. The system of claim 11 , wherein:

the tolerant logistic regression model includes a first and second logistic regression model;

the correct likelihood ratio is the first logistic regression model fit to each entry of the training data to minimize a likelihood function that each entry of the training data fits the first logistic regression model; and

the incorrect likelihood ratio is the second logistic regression model fit to each entry of the training data to minimize a likelihood function that each entry of the training data fits the second logistic regression model.

13. The system of claim 12 , wherein:

the correct probability for each entry is a multiplication or an addition of the correct likelihood ratio and the correct a priori odds ratio; and

the incorrect probability for each entry is a multiplication or an addition of the incorrect likelihood ratio and the incorrect a priori odds ratio.

14. A non-transitory machine computer-readable medium storing one or more sequences of instructions which when executed by one or more processors will cause the one or more processors to automatically carry out the steps of:

training a machine learning binary classifier using a set of training data entries, each training data entry having known inputs and a known output label, wherein the set of training data entries includes a subset of training data entries that have incorrectly labeled known output labels for both a binary true state and a binary false state of the output labels;

wherein training comprises:

determining a correct likelihood ratio that each training data entry of the set of training data entries has a correctly labeled output label and an incorrect likelihood ratio that each training data entry of the set of training data entries has an incorrectly labeled output label;

identifying a correct a priori odds ratio that the set of training data entries have correctly labeled output labels and an incorrect a priori odds ratio that the set of training data entries have incorrectly labeled output labels;

calculating a correct probability that each entry of the set of entries has a correctly labeled output label using the correct likelihood ratio for that entry and the correct a priori odds ratio;

calculating an incorrect probability that each entry of the set of entries has an incorrectly labeled output label using the incorrect likelihood ratio for that entry and the incorrect a priori odds ratio; and

training the machine learning binary classifier using a tolerant logistic regression model that combines the correct probability and the incorrect probability, wherein training the machine learning binary classifier using a logistic regression model includes adding the correct probability and the incorrect probability.

15. The non-transitory computer-readable medium of claim 14 , wherein:

the tolerant logistic regression model includes a first and second logistic regression model;

determining the correct likelihood ratio comprises fitting the first logistic regression model to each entry of the training data and determining the incorrect likelihood ratio comprises fitting the second logistic regression model to each entry of the training data; and

wherein fitting comprises minimizing a likelihood function that each entry of the training data fits a logistic regression model to estimate model parameters of a logistic regression model.

16. The non-transitory computer-readable medium of claim 14 , wherein:

calculating the correct probability for each entry is one of a multiplication or an addition of the correct likelihood ratio and the correct a priori odds ratio; and

calculating the incorrect probability for each entry is one of a multiplication or an addition of the incorrect likelihood ratio and the incorrect a priori odds ratio.