Machine learning methods and systems for identifying patterns in data using a plurality of learning machines wherein the learning machine that optimizes a performance function is selected
Methods for training machines to categorize data, and/or recognize patterns in data, and machines and systems so trained. More specifically, variations of the invention relates to methods for training machines that include providing one or more training data samples encompassing one or more data classes, identifying patterns in the one or more training data samples, providing one or more data samples representing one or more unknown classes of data, identifying patterns in the one or more of the data samples of unknown class(es), and predicting one or more classes to which the data samples of unknown class(es) belong by comparing patterns identified in said one or more data samples of unknown class with patterns identified in said one or more training data samples. Also provided are tools, systems, and devices, such as support vector machines (SVMs) and other methods and features, software implementing the methods and features, and computers or other processing devices incorporating and/or running the software, where the methods and features, software, and processors utilize specialized methods to analyze data.
1. A machine learning method comprising:
providing one or more training data samples having one or more known classes;
providing two or more learning machines, wherein said two or more learning machines comprise the same kernel type;
training said two or more learning machines to identify said one or more known classes using said one or more training data samples;
selecting the trained learning machine that optimizes a performance function dependent on one or more variables selected from the group consisting of maximizing divergence between the classes of data, n-fold cross validation, number of support vectors chosen, Vapnik-Chervonenkis (VC) dimension, ratio of all support vectors to unbounded support vectors, relative magnitude of the parameters concerned, favoring a large sigma for a Gaussian kernel, and small upper bound for Lagrange multipliers of each point, wherein test data is not used to select the trained learning machine; and
outputting the selected trained learning machine into a computer memory.
2. The method of claim wherein the selected trained learning machine is output on an output device.
3. The method of claim 1 , further comprising:
conducting a query of data contained in a database to identify data corresponding to said one or more known classes using the selected, trained learning machine; and
outputting data identified by the query as corresponding to said one or more known classes on an output device.
4. The method of claim 3 , wherein the data contained in the database have unknown classes, and the data are identified as corresponding to said one or more known classes by comparing patterns identified in said one or more unknown data samples with patterns identified in said one or more training data samples.
5. The method of claim 1 , wherein the one or more training data samples having one or more known classes are subjected to a feature reduction step prior to training to form training data samples each having differently reduced dimensionality, and wherein the two or more learning machines are trained using said training data samples having differently reduced dimensionality.
6. The method of claim 1 , wherein the learning machine comprises a sequential minimal optimization (SMO) algorithm, and each training data sample is assigned a weight within the SMO.
7. The method of claim 6 , wherein the weight is assigned to the training data sample by a user.
8. The method of claim 6 , wherein the weight is automatically assigned based on detection of quality measures within the training data sample.
9. The method of claim 1 , wherein the two or more learning machines are selected from the group consisting of a Support Vector Machine (SVM), a decision tree, a hidden Markov model, a Bayesian network, a Gram Schmidt algorithm, a neural network, and a genetic algorithm.
10. The method of claim 1 , wherein the one or more training data samples having one or more known classes contain supplemental data selected from the group consisting of normal data samples, negative data samples, randomly-generated data samples, theoretical data samples, and combinations thereof.
11. The method of claim 1 , wherein selecting the trained learning machine comprises the steps of providing a computer-readable program code means for comparing performance measures of the trained learning machines and predicting querying success.
12. The method of claim 1 , wherein the one or more training data samples having one or more known classes are subjected to a feature reduction step prior to training to form training data samples each having differently reduced dimensionality, and wherein the two or more learning machines are trained using said training data samples having differently reduced dimensionality.
13. A machine learning method comprising:
providing one or more data patterns;
providing one or more data samples;
providing two or more learning machines, wherein said two or more learning machines comprise the same kernel type;
training said two or more learning machines to identify which of the one or more data samples correspond to the one or more data patterns;
selecting the trained learning machine that identifies which of the one or more data samples correspond to the one or more data patterns by optimizing a performance function dependent on one or more variables selected from the group consisting of maximizing divergence between the classes of data, n-fold cross validation, number of support vectors chosen, Vapnik-Chervonenkis (VC) dimension, ratio of all support vectors to unbounded support vectors, relative magnitude of the parameters concerned, favoring a large sigma for a Gaussian kernel, and small upper bound for Lagrange multipliers of each point, wherein no test data set is used to select the trained learning machine; and
outputting the selected trained learning machine into a computer memory.
14. The method of claim 13 , wherein the selected trained learning machine is output on an output device.
15. The method of claim 13 , further comprising:
conducting a query of data contained in a database to identify data corresponding to said one or more data patterns using the selected, trained learning machine; and
outputting data identified by the query as corresponding to said one or more data patterns on an output device.
16. The method of claim 15 , wherein the data are identified as corresponding to said one or more data patterns by comparing said one or more unknown data samples with patterns identified in said one or more data samples.
17. The method of claim 13 , wherein the one or more data samples are subjected to a feature reduction step prior to training to form training data samples having differently reduced dimensionality, and wherein the two or more learning machines are trained using said training data samples having differently reduced dimensionality.
18. The method of claim 13 , wherein the two or more learning machines are selected from the group consisting of a Support Vector Machine (SVM), a decision tree, a hidden Markov model, a Bayesian network, a Gram Schmidt algorithm, a neural network, a genetic algorithm, and a sequential minimal optimization (SMO) algorithm.
19. A computer program product comprising a computer usable medium having control logic stored therein for causing a computer to produce a trained learning machine, the control logic comprising:
first computer readable program code means for providing one or more training data samples having one or more known classes;
second computer readable program code means for training two or more learning machines, wherein the two or more learning machines comprise the same kernel type, using said one or more training data samples;
third computer readable program code means for selecting the trained learning machine that optimizes a performance function dependent on one or more variables selected from the group consisting of maximizing divergence between the classes of data, n-fold cross validation, number of support vectors chosen, Vapnik-Chervonenkis (VC) dimension, ratio of all support vectors to unbounded support vectors, relative magnitude of the parameters concerned, favoring a large sigma for a Gaussian kernel, and small upper bound for Lagrange multipliers of each point, wherein no test data set is used to select the trained learning machine; and
fourth computer readable program code means for outputting the selected trained learning machine on an output device.
20. The computer program product of claim 19 , wherein the computer usable medium is provided in a computer system.