Blackbox optimization via model ensembling
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for optimizing a plurality of parameters of a black box system with respect to a distribution of the plurality of parameters are disclosed. Methods can include obtaining a fitness function of the black box system and a training dataset including multiple training samples. Methods can further include training an ensemble of machine learning model wherein during each iteration of the training process an output of the system is generated for each of the multiple training samples. During each iteration, a predictive value is also generated using an ensemble of machine learning models. A natural gradient is determined based on the output of the system and the predictive value. The ensemble of machine learning models is then trained using the natural gradient.
1 . A computer-implemented method comprising:
obtaining a training dataset comprising multiple training samples, wherein each training example comprises a plurality of features;
performing, by one or more computing devices, one or more iterations of a sequence of operations, comprising:
generating, for each of the multiple training samples and using an ensemble of machine learning models, a predicted model output value;
for each predicted model output value, perturbing, two or more times, the predicted model output value by adding noise, resulting in multiple perturbed predicted values for each predicted model output value;
determining a natural gradient for each of the multiple training samples based on the multiple perturbed predicted values and an output of a black box system, wherein the natural gradient is determined in a space of dependent values decoupled from model parameterization;
training the ensemble of machine learning models using the training dataset and the natural gradient determined based on the multiple perturbed predicted values and the output of the black box system; and
selecting, using the trained ensemble of machine learning models, a set of black box system parameters that optimizes the black box system.
2 . The computer-implemented method of claim 1 , wherein perturbing, two or more times, the predicted model output value is performed based on a zero-centered Gaussian distribution parameterized by standard deviation σ.
3 . The computer-implemented method of claim 1 , wherein the ensemble of machine learning models comprises Gradient Boosted Trees.
4 . The computer-implemented method of claim 1 , wherein: training the ensemble of machine learning models comprises training an ensemble of ranking models that rank items using features of an input query.
5 . The computer-implemented method of claim 1 , wherein training the ensemble of machine learning models comprises training an ensemble of counterfeiting models that output a probability of an item being a counterfeit based on features of a candidate item input to the ensemble of counterfeiting models.
6 . A system, comprising:
a memory device; and
one or more processors configured to interact with the memory device and configured to perform operations, including:
obtaining a training dataset comprising multiple training samples, wherein each training example comprises a plurality of features;
performing one or more iterations of a sequence of operations, comprising:
generating, for each of the multiple training samples and using an ensemble of machine learning models, a predicted model output value;
for each predicted model output value, perturbing, two or more times, the predicted model output value by adding noise, resulting in multiple perturbed predicted values for each predicted model output value;
determining a natural gradient for each of the multiple training samples based on the multiple perturbed predicted values and an output of a black box system, wherein the natural gradient is determined in a space of dependent values decoupled from model parameterization;
training the ensemble of machine learning models using the training dataset and the natural gradient determined based on the multiple perturbed predicted values and the output of the black box system; and
selecting, using the trained ensemble of machine learning models, a set of black box system parameters that optimizes the black box system.
7 . The system of claim 6 , wherein perturbing, two or more times, the predicted model output value is performed based on a zero-centered Gaussian distribution parameterized by standard deviation σ.
8 . The system of claim 6 , wherein the ensemble of machine learning models comprises Gradient Boosted Trees.
9 . The system of claim 6 , wherein:
training the ensemble of machine learning models comprises training an ensemble of ranking models that rank items using features of an input query.
10 . The system of claim 6 , wherein training the ensemble of machine learning models comprises training an ensemble of counterfeiting models that output a probability of an item being a counterfeit based on features of a candidate item input to the ensemble of counterfeiting models.
11 . A non-transitory computer readable medium storing instructions that, when executed by one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising:
obtaining a training dataset comprising multiple training samples, wherein each training example comprises a plurality of features;
performing, by one or more computing devices, one or more iterations of a sequence of operations, comprising:
generating, for each of the multiple training samples and using an ensemble of machine learning models, a predicted model output value;
for each predicted model output value, perturbing, two or more times, the predicted model output value by adding noise, resulting in multiple perturbed predicted values for each predicted model output value;
determining a natural gradient for each of the multiple training samples based on the multiple perturbed predicted values and an output of a black box system, wherein the natural gradient is determined in a space of dependent values decoupled from model parameterization;
training the ensemble of machine learning models using the training dataset and the natural gradient determined based on the multiple perturbed predicted values and the output of the black box system; and
selecting, using the trained ensemble of machine learning models, a set of black box system parameters that optimizes the black box system.
12 . The non-transitory computer readable medium of claim 11 , wherein perturbing, two or more times, the predicted model output value is performed based on a zero-centered Gaussian distribution parameterized by standard deviation σ.
13 . The non-transitory computer readable medium of claim 11 , wherein the ensemble of machine learning models comprises Gradient Boosted Trees.
14 . The non-transitory computer readable medium of claim 11 , wherein training the ensemble of machine learning models comprises training an ensemble of ranking models that rank items using features of an input query.
15 . The non-transitory computer readable medium of claim 11 , wherein training the ensemble of machine learning models comprises training an ensemble of counterfeiting models that output a probability of an item being a counterfeit based on features of a candidate item input to the ensemble of counterfeiting models.