Locating a decision boundary for complex classifier
Systems and methods improve performance of a classifier, which comprises a neural network and is trained through machine learning. First and second scores are computed, by the classifier, for each a multiple data examples from a generator. The first score is indicative of whether the data example belongs to a first data cluster and the second score is indicative of whether the data example belongs to a second data cluster. The generator is trained with an objective such that, for each data example generated by the generator, the first and second scores computed by the classifier are equal. Partial derivatives from the classifier are back-propagated for multiple data examples generated by the generator, to obtain a vector, for each data example, that is orthogonal to a decision surface for the classifier. A problem with the classifier is detected based on changes in directions of the vectors. Upon detecting a problem, the classifier is adjusted to reduce errors by the classifier caused by overfitting training data.
1. A method for improving performance of a classifier, wherein the classifier comprises a deep neural network that is trained iteratively through machine learning, wherein the deep neural network of the classifier comprises an input layer, an output layer, and one or more inner layers, wherein the layers comprise interconnected nodes that process and transform input data through weighted connections, the method comprising:
computing, by the classifier implemented by a computer system, first and second scores for each of multiple data examples from a generator, wherein the first score is indicative of whether the data example belongs to a first data cluster and the second score is indicative of whether the data example belongs to a second data cluster, wherein the generator is trained with an objective such that, for each data example generated by the generator, the first and second scores computed by the classifier are equal;
back-propagating, by the computer system, for multiple data examples generated by the generator, partial derivatives from the classifier to obtain a vector, for each data example, that is orthogonal to a decision surface for the classifier;
detecting, by the computer system, a problem with the classifier based on changes in directions of the vectors; and
upon detecting a problem, adjusting, by the computer system, the classifier to reduce errors by the classifier caused by overfitting training data.
2. The method of claim 1 , further comprising training, through machine learning, by the computer system, the generator to generate the multiple data examples.
3. The method of claim 1 , further comprising, prior to back-propagating partial derivatives from the classifier, training the classifier to discriminate between the first and second data clusters.
4. The method of claim 3 , wherein training the classifier to discriminate between the first and second data clusters comprises training, by the computer system, the classifier to imitate an original machine-learning discriminator that does not comprise a neural network.
5. The method of claim 1 , wherein detecting the problem comprises detecting, by the computer system, changes in the directions of the vectors that are greater than a threshold value.
6. The method of claim 1 , wherein adjusting the classifier comprises further training, by the computer system, the classifier with a revised hyperparameter.
7. The method of claim 6 , wherein adjusting the classifier further comprises measuring, by the computer system, changes in the first and second scores for the further training with the revised hyperparameter.
8. The method of claim 1 , wherein adjusting the classifier comprises changing, by the computer system, an architecture of the classifier.
9. The method of claim 1 , wherein adjusting the classifier comprises substituting, by the computer system, a new machine learning system for the classifier, wherein the new machine learning system is trained on different data than a prior training of the classifier.
10. The method of claim 1 , wherein the first data cluster corresponds to a first classification category and the second data cluster corresponds to a second classification category.
11. The method of claim 1 , wherein adjusting the classifier comprises further training, by the computer system, the classifier with additional data examples near the decision surface.
12. The method of claim 11 , further comprising generating, by the computer system, the additional data examples with a stochastic autoencoder.
13. The method of claim 2 , wherein the classifier computes the first and second scores using a softmax function.
14. The method of claim 13 , wherein training the generator comprises training the generator such that the both the first and second scores are 0.5 with the softmax function.
15. The method of claim 1 , further comprising generating, by the computer system, a characterization of the decision surface.
16. The method of claim 15 , wherein generating the characterization of the decision surface comprises fitting, by the computer system, a hyperplane to the multiple data examples.
17. A computer system for improving performance of a classifier, wherein the classifier comprises a deep neural network that is trained iteratively through machine learning, wherein the deep neural network of the classifier comprises an input layer, an output layer, and one or more inner layers, wherein the layers comprise interconnected nodes that process and transform input data through weighted connections, the computer system comprising:
one or more processor cores; and
computer memory in communication with the one or more processor cores, wherein the computer memory stores instructions that when executed by the one or more processor cores, cause the one or more processor cores to:
compute, by the classifier, first and second scores for each of multiple data examples from a generator, wherein the first score is indicative of whether the data example belongs to a first data cluster and the second score is indicative of whether the data example belongs to a second data cluster, wherein the generator is trained with an objective such that, for each data example generated by the generator, the first and second scores computed by the classifier are equal;
back-propagate, for multiple data examples generated by the generator, partial derivatives from the classifier to obtain a vector, for each data example, that is orthogonal to a decision surface for the classifier; and
detect a problem with the classifier based on changes in directions of the vectors; and
upon detecting a problem, adjust the classifier to reduce errors by the classifier caused by overfitting training data.
18. The computer system of claim 17 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to train, through machine learning, the generator to generate the multiple data examples.
19. The computer system of claim 17 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to, prior to back-propagating partial derivatives from the classifier, train the classifier to discriminate between the first and second data clusters.
20. The computer system of claim 19 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to train the classifier to discriminate between the first and second data clusters by training the classifier to imitate an original machine-learning discriminator that does not comprise a neural network.
21. The computer system of claim 17 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to detect the problem by detecting changes in the directions of the vectors that are greater than a threshold value.
22. The computer system of claim 17 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to adjust the classifier by further training the classifier with a revised hyperparameter.
23. The computer system of claim 22 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to adjust the classifier by measuring changes in the first and second scores for the further training with the revised hyperparameter.
24. The computer system of claim 17 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to adjust the classifier by changing an architecture of the classifier.
25. The computer system of claim 17 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to adjust the classifier by substituting a new machine learning system for the classifier, wherein the new machine learning system is trained on different data than a prior training of the classifier.
26. The computer system of claim 17 , wherein the first data cluster corresponds to a first classification category and the second data cluster corresponds to a second classification category.
27. The computer system of claim 17 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to adjust the classifier by further training the classifier with additional data examples near the decision surface.
28. The computer system of claim 27 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to generate the additional data examples with a stochastic autoencoder.
29. The computer system of claim 18 , wherein the classifier computes the first and second scores using a softmax function.
30. The computer system of claim 29 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to train the generator by training the generator such that the both the first and second scores are 0.5 with the softmax function.
31. The computer system of claim 17 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to generate a characterization of the decision surface.
32. The computer system of claim 31 , wherein the computer memory stores further instructions that when executed by the one or more processing cores, cause the one or more processor cores to generate the characterization of the decision surface comprises fitting a hyperplane to the multiple data examples.
33. The method of claim 2 , wherein:
the method further comprises, prior to back-propagating partial derivatives from the classifier, training the classifier to discriminate between the first and second data clusters;
training the generator comprises training the generator with a first set of one or more processor cores of the computer system; and
training the classifier comprising training the classifier with a second set of one or more processor cores of the computer system.
34. The method of claim 33 , wherein the classifier comprises an image classifier.
35. The method of claim 33 , wherein the classifier comprises a diagnostic classifier.
36. The method of claim 33 , wherein:
the first set of one or more processor cores comprises a first graphics processing unit; and
the second set of one or more processor cores comprises a second graphics processing unit.