Simultaneous data sampling and feature selection via weak learners
From many features and many multidimensional points, a computer generates exploratory training configurations. Each point contains a value for each of the features. Each exploratory training configuration identifies a random subset of the features and a random subset of the points. A performance score is generated for each of the exploratory training configurations. A feature weight is generated for each of the features that is based on the performance scores of the exploratory training configurations whose random subset of features contains the feature. A point weight is generated for each of the points that is based on the performance scores of the exploratory training configurations whose random subset of the many points contains the point. A machine learning model is trained using an optimized training corpus that consists of a subset of the many features based on feature weight and a subset of the many points based on point weight.
1 . A method comprising:
generating a training corpus by:
a) generating, from a plurality of features and a plurality of multidimensional points, a plurality of training configurations wherein:
each point in the plurality of multidimensional points contains a value for each feature in the plurality of features, and
each configuration in the plurality of training configurations identifies a random subset of the plurality of features and a random subset of the plurality of multidimensional points;
b) generating a performance score for each configuration in the plurality of training configurations by training and evaluating a weak learner based on the random subset of the plurality of features of the configuration and the random subset of the plurality of multidimensional points of the configuration;
c) generating a feature weight for each feature in the plurality of features that is based on the performance scores of the plurality of training configurations whose random subset of the plurality of features contains the feature;
d) generating a point weight for each point in the plurality of multidimensional points that is based on the performance scores of the plurality of training configurations whose random subset of the plurality of multidimensional points contains the point; and
e) including in the training corpus:
a subset of the plurality of features having feature weights that exceed a first threshold, and
a subset of the plurality of multidimensional points having point weights that exceed a second threshold; and
training, with the training corpus, a machine learning model that does not comprise a weak learner;
wherein the method is performed by one or more computers.
2 . The method of claim 1 wherein:
the weak learner comprises a decision tree; and
the machine learning model does not comprise a decision tree.
3 . The method of claim 1 further comprising for each feature in the plurality of features, counting how many of the plurality of training configurations whose random subset of the plurality of features of the configuration contains the feature.
4 . The method of claim 1 further comprising for each point in the plurality of multidimensional points, counting how many of the plurality of training configurations whose random subset of the plurality of multidimensional points of the configuration contains the point.
5 . The method of claim 1 wherein said generating the plurality of training configurations comprises generating a fixed count of training configurations.
6 . The method of claim 1 further comprising for each feature in the plurality of features, summing performance scores of the plurality of training configurations whose random subset of the plurality of features of the configuration contains the feature.
7 . The method of claim 1 further comprising for each point in the plurality of multidimensional points, summing performance scores of the plurality of training configurations whose random subset of the plurality of multidimensional points of the configuration contains the point.
8 . The method of claim 1 wherein at least one selected from a group consisting of:
said generating the plurality of training configurations occurs before said generating the performance score for each configuration in the plurality of training configurations, and
said generating the plurality of training configurations does not depend on said generating the performance score for each configuration in the plurality of training configurations.
9 . The method of claim 1 wherein said training the machine learning model comprises the machine learning model accepting as input a point weight of a point in the plurality of multidimensional points.
10 . One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause:
generating a training corpus by:
a) generating, from a plurality of features and a plurality of multidimensional points, a plurality of training configurations wherein:
each point in the plurality of multidimensional points contains a value for each feature in the plurality of features, and
each configuration in the plurality of training configurations identifies a random subset of the plurality of features and a random subset of the plurality of multidimensional points;
b) generating a performance score for each configuration in the plurality of training configurations by training and evaluating a weak learner based on the random subset of the plurality of features of the configuration and the random subset of the plurality of multidimensional points of the configuration;
c) generating a feature weight for each feature in the plurality of features that is based on the performance scores of the plurality of training configurations whose random subset of the plurality of features contains the feature;
d) generating a point weight for each point in the plurality of multidimensional points that is based on the performance scores of the plurality of training configurations whose random subset of the plurality of multidimensional points contains the point; and
e) including in the training corpus:
a subset of the plurality of features having feature weights that exceed a first threshold, and
a subset of the plurality of multidimensional points having point weights that exceed a second threshold; and
training, with the training corpus, a machine learning model that does not comprise a weak learner.
11 . The one or more non-transitory computer-readable media of claim 10 wherein:
the weak learner comprises a decision tree; and
the machine learning model does not comprise a decision tree.
12 . The one or more non-transitory computer-readable media of claim 10 wherein the instructions further cause for each feature in the plurality of features, counting how many of the plurality of training configurations whose random subset of the plurality of features of the configuration contains the feature.
13 . The one or more non-transitory computer-readable media of claim 10 wherein the instructions further cause for each point in the plurality of multidimensional points, counting how many of the plurality of training configurations whose random subset of the plurality of multidimensional points of the configuration contains the point.
14 . The one or more non-transitory computer-readable media of claim 10 wherein said generating the plurality of training configurations comprises generating a fixed count of training configurations.
15 . The one or more non-transitory computer-readable media of claim 10 wherein the instructions further cause for each feature in the plurality of features, summing performance scores of the plurality of training configurations whose random subset of the plurality of features of the configuration contains the feature.
16 . The one or more non-transitory computer-readable media of claim 10 wherein the instructions further cause for each point in the plurality of multidimensional points, summing performance scores of the plurality of training configurations whose random subset of the plurality of multidimensional points of the configuration contains the point.
17 . The one or more non-transitory computer-readable media of claim 10 wherein at least one selected from a group consisting of:
said generating the plurality of training configurations occurs before said generating the performance score for each configuration in the plurality of training configurations, and
said generating the plurality of training configurations does not depend on said generating the performance score for each configuration in the plurality of training configurations.
18 . The one or more non-transitory computer-readable media of claim 10 wherein said training the machine learning model comprises the machine learning model accepting as input a point weight of a point in the plurality of multidimensional points.