Algorithm-specific neural network architectures for automatic machine learning model selection

1. A method comprising:

deriving a plurality of meta-feature values that describe an inference dataset as a whole by, for each meta-feature of a plurality of meta-features, deriving a respective meta-feature value from an inference dataset;

for each algorithm of a plurality of trainable algorithms:

for each algorithm-specific meta-model of a plurality of algorithm-specific regression meta-models that are respectively associated with the algorithm, predicting a respective score by invoking the algorithm-specific meta-model based on a) a respective subset of meta-feature values of the plurality of meta-feature values, and b) hyperparameter values of a respective subset of a plurality of hyperparameters of the algorithm, wherein the respective score indicates at least one selected from the group consisting of: an accuracy of the algorithm from training with the inference dataset and a duration needed to train the algorithm with the inference dataset;

selecting, based on a comparison of the respective scores of the plurality of algorithm-specific regression meta-models, one or more algorithms of the plurality of trainable algorithms; and

training and invoking, based on the inference dataset, the one or more algorithms to obtain a result;

wherein the method is performed by one or more computers.

2. The method of claim 1 wherein:

selecting the one or more algorithms comprises:

ranking the plurality of trainable algorithms based on the respective scores, and selecting multiple algorithms based on the ranking.

3. The method of claim 1 wherein said hyperparameter values are default values.

4. The method of claim 1 wherein:

each algorithm of the plurality of trainable algorithms is associated with a respective algorithm-specific ensemble of a plurality of algorithm-specific ensembles that contains said plurality of algorithm-specific regression meta-models that are associated with the algorithm;

the method further comprises deriving, for each algorithm of the plurality of trainable algorithms, a respective ensemble score that is based on the respective scores of the plurality of algorithm-specific regression meta-models that are associated with the algorithm;

selecting the one or more algorithms based on the respective scores comprises selecting the one or more algorithms based on the respective ensemble scores.

5. The method of claim 1 further comprising:

storing a plurality of testing datasets;

for each algorithm of the plurality of trainable algorithms:

for each model of a plurality of models that are based on the algorithm:

configuring the model based on respective particular values for said plurality of hyperparameters of the algorithm; and

for each testing dataset of the plurality of testing datasets:

testing the model based on the testing dataset to calculate a respective test score; and

recording a distinct tuple that references: the respective particular values for said plurality of hyperparameters, the testing dataset, the respective test score, and the algorithm; and

for each algorithm-specific meta-model of the plurality of algorithm-specific regression meta-models that are associated with the algorithm, training the algorithm-specific meta-model based on at least one of said distinct tuples recorded for the algorithm.

6. The method of claim 5 wherein said plurality of meta-features excludes meta-features that are missing a value in a percentage of the distinct tuples that exceeds a threshold.

7. The method of claim 5 further comprising cross validating the plurality of models with an original training dataset that is partitioned into: a plurality of training datasets and said plurality of testing datasets.

8. The method of claim 5 further comprising generating, in parallel, multiple tuples of said distinct tuples.

9. The method of claim 1 wherein the plurality of trainable algorithms comprises a random forest.

10. The method of claim 1 wherein the plurality of trainable algorithms comprises anomaly detection.

11. The method of claim 1 wherein each algorithm-specific meta-model of the plurality of algorithm-specific regression meta-models comprises a distinct artificial neural network.

12. The method of claim 1 further comprising applying a softmax function to the respective scores of the plurality of algorithm-specific regression meta-models.

13. The method of claim 1 further comprising converting values of a non-numeric meta-feature of said plurality of meta-features by an encoding scheme of one-cold.

14. The method of claim 1 further comprising converting values of a numeric meta-feature of said plurality of meta-features by at least one encoding scheme of: zero mean or unit variance.

15. The method of claim 1 further comprising, for each algorithm of the plurality of trainable algorithms, optimizing

a count of algorithm-specific meta-models in the plurality of algorithm-specific regression meta-models that are associated with the algorithm.

16. The method of claim 15 wherein said optimizing comprises using at least one selected from the group consisting of: Bayesian optimization and a decision tree.

17. The method of claim 1 further comprising assigning meta-features as inputs for each algorithm-specific meta-model of the plurality of algorithm-specific regression meta-models that are associated with each algorithm of the plurality of trainable algorithms by at least one selected from the group consisting of: sample bagging and feature bagging.

18. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause:

deriving a plurality of meta-feature values that describe an inference dataset as a whole by, for each meta-feature of a plurality of meta-features, deriving a respective meta-feature value from an inference dataset;

for each algorithm of a plurality of trainable algorithms:

for each algorithm-specific meta-model of a plurality of algorithm-specific regression meta-models that are respectively associated with the algorithm, predicting a respective score by invoking the algorithm-specific meta-model based on a) a respective subset of meta-feature values of the plurality of meta-feature values, and b) hyperparameter values of a respective subset of a plurality of hyperparameters of the algorithm, wherein the respective score indicates at least one selected from the group consisting of: an accuracy of the algorithm from training with the inference dataset and a duration needed to train the algorithm with the inference dataset;

selecting, based on a comparison of the respective scores of the plurality of algorithm-specific regression meta-models, one or more algorithms of the plurality of trainable algorithms; and

training and invoking, based on the inference dataset, the one or more algorithms to obtain a result.

19. The one or more non-transitory computer-readable media of claim 18 wherein:

each algorithm of the plurality of trainable algorithms is associated with a respective algorithm-specific ensemble of a plurality of algorithm-specific ensembles that contains said plurality of algorithm-specific regression meta-models that are associated with the algorithm;

the instructions further cause deriving, for each algorithm of the plurality of trainable algorithms, a respective ensemble score that is based on the respective scores of the plurality of algorithm-specific regression meta-models that are associated with the algorithm;

selecting the one or more algorithms based on the respective scores comprises selecting the one or more algorithms based on the respective ensemble scores.

20. The one or more non-transitory computer-readable media of claim 18 wherein the instructions further cause:

storing a plurality of testing datasets;

for each algorithm of the plurality of trainable algorithms:

for each model of a plurality of models that are based on the algorithm:

configuring the model based on respective particular values for said plurality of hyperparameters of the algorithm; and

for each testing dataset of the plurality of testing datasets:

testing the model based on the testing dataset to calculate a respective test score; and

recording a distinct tuple that references: the respective particular values for said plurality of hyperparameters, the testing dataset, the respective test score, and the algorithm; and

for each algorithm-specific meta-model of the plurality of algorithm-specific regression meta-models that are associated with the algorithm, training the algorithm-specific meta-model based on at least one of said distinct tuples recorded for the algorithm.