Methods for feature selection using classifier ensemble based genetic algorithms
Methods for performing genetic algorithm-based feature selection are provided herein. In certain embodiments, the methods include steps of applying multiple data splitting patterns to a learning data set to build multiple classifiers to obtain at least one classification result; integrating the at least one classification result from the multiple classifiers to obtain an integrated accuracy result; and outputting the integrated accuracy result to a genetic algorithm as a fitness value for a candidate feature subset, in which genetic algorithm-based feature selection is performed.
1. A method for performing genetic algorithm-based feature selection, the method comprising:
dividing a set of data samples into at least a first sub-set of data samples and a second sub-set of data samples, wherein the first sub-set of data samples and the second sub-set of data samples include different sub-sets of the set of data samples;
applying a data splitting pattern to the first sub-set of data samples, splitting the first sub-set of data samples into a first training set of data samples and a first testing set of data samples, wherein the first training set of data samples and the first testing set of data samples include different data samples of the first sub-set of data samples;
creating a first classifier based on the first training set of data samples and the first testing set of data samples;
applying one or more different data splitting patterns to the first sub-set of data samples, splitting the first sub-set of data samples into one or more training sets of data samples and one or more testing sets of data samples, wherein the one or more training sets of data samples and the corresponding one or more testing sets of data samples include different data samples of the first sub-set of data samples;
creating one or more classifiers based on the one or more training sets of data samples and the corresponding one or more testing sets of data samples;
integrating the first classifier and the one or more classifiers, generating an integrated classifier; and
outputting the integrated classifier to a genetic algorithm as a fitness value for a candidate feature subset, wherein genetic algorithm-based feature selection is performed using the integrated classifier.
2. The method according to claim 1 , further comprising using the genetic algorithm to obtain the candidate feature subset.
3. The method according to claim 1 , wherein the first classifier and the one or more classifiers are selected from the group consisting of at least one of a support vector machine, a decision tree, linear discriminant analysis, and a neural network.
4. The method according to claim 1 , further comprising:
using a re-sampling technique to obtain the first training and the one or more training sets of data samples and the first testing and the one or more testing sets of data samples from the first sub-set of data samples.
5. The method according to claim 1 , further comprising combining classification results from the first classifier and the one or more classifiers to form a group prediction.
6. The method according to claim 1 , wherein integrating the first classifier and the one or more classifiers further comprises calculating at least one result selected from a group consisting of an average, a weighted average, and a weighted majority vote.
7. The method according to claim 1 , further comprising using the genetic algorithm to obtain an optimal final feature subset.
8. The method according to claim 1 , wherein the method is used in a medical imaging modality selected from a group consisting of at least one of CT, MRI, X-ray and ultrasound.
9. The method according to claim 1 , wherein the method is used in computer aided detection (CAD).
10. The method according to claim 9 , wherein the method is used in CAD of a disease selected from a group consisting of at least one of lung cancer, breast cancer, prostate cancer and colorectal cancer.
11. The method according to claim 1 , wherein the method is used in computer aided diagnosis (CADx).
12. The method according to claim 11 , wherein the method is used in CADx of a disease selected from a group consisting of at least one of lung cancer, breast cancer, prostate cancer and colorectal cancer.
13. The method according to claim 1 , further comprising:
evaluating the integrated classifier by performing an accuracy of classification of the integrated classifier using the second sub-set of data samples;
generating an evaluation result, which are the integrated results from each classifier; and
outputting the evaluation result to the genetic algorithm as the fitness value for the candidate feature subset.
14. The method according to claim 1 , wherein the fitness value includes specificity.
15. The method according to claim 1 , wherein the fitness value includes sensitivity.
16. The method according to claim 1 , wherein the fitness value includes both specificity and sensitivity.
17. The method according to claim 1 , further comprising:
determining, with the genetic algorithm, features to retain and features to discard.
18. The method according to claim 1 , further comprising:
generating, with the genetic algorithm, a subsequent set of data samples through mutation and crossover operations.
19. The method according to claim 18 , further comprising:
splitting the subsequent set of data samples into at least a first sub-set of the subsequent set of data samples and a second sub-set of the subsequent set of data samples, wherein the first sub-set of the subsequent set of data samples and the second sub-set of the subsequent set of data samples include different sub-sets of the subsequent set of data samples;
applying a subsequent data splitting pattern to the first sub-set of the subsequent set of data samples, splitting the first sub-set of the subsequent set of data samples into a first training set of the subsequent set of data samples and a first testing set of the subsequent set of data samples, wherein the first training set of the subsequent set of data samples and the first testing set of the subsequent set of data samples include different data samples of the first sub-set of the subsequent set of data samples;
creating a subsequent first classifier based on the first training set of the subsequent set of data samples and the first testing set of the subsequent set of data samples;
applying one or more different subsequent data splitting patterns to the first sub-set of the subsequent set of data samples, splitting the first sub-set of the subsequent set of data samples into one or more subsequent training sets of the subsequent set of data samples and one or more subsequent testing sets of the subsequent set of data samples, wherein the one or more subsequent training sets of the subsequent set of data samples and the corresponding one or more subsequent testing sets of the subsequent set of data samples include different data samples of the first sub-set of the subsequent set of data samples;
creating one or more subsequent classifiers based on the one or more training sets of the subsequent set of data samples and the corresponding one or more subsequent testing sets of the subsequent set of data samples;
integrating the first subsequent classifier and the one or more subsequent classifiers, generating a subsequent integrated classifier; and
outputting the subsequent integrated classifier to the genetic algorithm as the fitness value for the candidate feature subset, wherein subsequent genetic algorithm-based feature selection is performed using the subsequent integrated classifier.