Method of generating features optimal to a dataset and classifier
A method of generating features optimal to a particular dataset and classifier is disclosed. A dataset of messages is inputted and a classifier is selected. An algebra of features is encoded. Computable features that are capable of describing the dataset from the algebra of features are selected. Irredundant features that are optimal for the classifier and the dataset are selected.
1. A method of generating irredundant features optimal to a particular dataset and classifier, comprising:
a. inputting a dataset of messages;
b. inputting a classifier;
c. encoding an algebra of computable features capable of describing the dataset through expression trees; and
d. applying a genetic algorithm to select a resulting feature which maximizes accuracy of the classifier given a validation set.
2. The method of claim 1 wherein the resulting feature is cached and the algorithm iteratively reapplied to determine a set of irredundant features maximizing classifier accuracy for the dataset, wherein when computing accuracy the classifier utilizes the cached features and a candidate feature.
3. The method of claim 1 wherein the trees are specified by a BNF grammar or a Chomsky grammar.
4. The method of claim 1 further comprising randomly selecting a first subset of the algebra of computable features and inputting them into the classifier.
5. The method of claim 4 further comprising randomly selecting a second subset of features from the first subset of computable features.
6. The method of claim 5 further comprising mutating the second subset of features.
7. The method of claim 6 further comprising randomly selecting pairs of features from the first subset of computable features.
8. The method of claim 7 further comprising applying crossing to each selected pair of features from the first subset of computable features.
9. The method of claim 8 further comprising combining the first subset of computable features, the mutated features, and the crossed features to generate a third subset of features.
10. The method of claim 9 further comprising selecting a fourth set of features from the algebra of features and adding to the third subset of features.
11. The method of claim 10 further comprising removing duplicate features from the third subset of features.
12. The method of claim 11 further comprising inputting the features from the third subset into the classifier and computing accuracy of the classifier.
13. The method of claim 12 further comprising ranking the features of the third subset by their corresponding accuracy.
14. The method of claim 13 further comprising breaking ties, if any, in the ranking based on a total order.
15. The method of claim 14 further comprising maintaining only fittest features from the third subset to form a fifth subset of the same size as the first subset, and replacing the first subset with the fifth subset.
16. The method of claim 15 further comprising repeating the mutating, the crossing, the removing, the ranking, the selecting, and the maintaining until accuracy of a most fit feature converges, wherein the most fit feature is cached.