Systems and methods implementing an intelligent optimization platform
A disclosed example includes implementing a first worker instance and a second worker instance to operate in parallel; running a first tuning operation via the first worker instance to tune first hyperparameters; running a second tuning operation via the second worker instance using a Bayesian-based optimization to determine a hyperparameter configuration to evaluate next; evaluating the hyperparameter configuration for an external model using a surrogate model; and selecting the hyperparameter configuration for the external model.
1. An apparatus comprising:
at least one memory;
machine-readable instructions in the apparatus; and
at least one processor circuit to execute the machine-readable instructions to:
cause a first machine instance and a second machine instance to operate in parallel to service a work request;
cause the first machine instance to run a first tuning operation to generate a first hyperparameter configuration;
cause the second machine instance to run a second tuning operation to generate a second hyperparameter configuration;
before completion of the work request:
evaluate the first hyperparameter configuration and the second hyperparameter configuration for a first model using a surrogate model of the first model;
generate a first probability that the first hyperparameter configuration improves a performance of a computer that is to execute the first model, and generate a second probability that the second hyperparameter configuration improves the performance of the computer that is to execute the first model; and
generate a ranking of the first hyperparameter configuration and the second hyperparameter configuration based on the first probability and the second probability; and
cause transmission of a partial response to the computer, the partial response to include:
the ranking of the first hyperparameter configuration and the second hyperparameter configuration;
the first probability;
the second probability; and
an indication of time remaining to generate a full response based on the completion of the work request.
2. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to evaluate the first hyperparameter configuration by testing an efficacy of the first hyperparameter configuration against at least one objective function of the first model.
3. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to:
estimate a structure of the first model based on at least one or more objective functions of the first model; and
generate the surrogate model based on the structure.
4. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to cause storing of a level of completion of the work request based on the first machine instance and the second machine instance.
5. The apparatus of claim 1 , wherein the first model is a machine learning model.
6. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to cause the first machine instance to run the first tuning operation based on one or more tuning constraints.
7. The apparatus of claim 1 , wherein, during servicing of the work request, one or more of the at least one processor circuit is to cause the transmission of the partial response after a request from the computer for a partial completion of the work request, the request for the partial completion of the work request received after the work request.
8. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to generate the surrogate model as an approximation of the first model.
9. At least one of a random-access memory (RAM), a read-only memory (ROM), a flash memory, an electrically erasable programmable ROM (EEPROM), an optical device, or a hard drive comprising instructions to cause at least one processor circuit to at least:
cause a first machine instance and a second machine instance to operate in parallel to service a work request;
cause the first machine instance to run a first tuning operation to generate a first hyperparameter configuration;
cause the second machine instance to run a second tuning operation to generate a second hyperparameter configuration;
before completion of the work request:
evaluate the first hyperparameter configuration and the second hyperparameter configuration for a first model using a surrogate model of the first model;
generate a first probability that the first hyperparameter configuration improves a performance of a computer that is to execute the first model, and generate a second probability that the second hyperparameter configuration improves the performance of the computer that is to execute the first model; and
generate a ranking of the first hyperparameter configuration and the second hyperparameter configuration based on the first probability and the second probability; and
cause transmission of a partial response to the computer, the partial response to include:
the ranking of the first hyperparameter configuration and the second hyperparameter configuration;
the first probability;
the second probability; and
an indication of time remaining to generate a full response based on the completion of the work request.
10. The at least one of the RAM, the ROM, the flash memory, the EEPROM, the optical device, or the hard drive of claim 9 , wherein the instructions are to cause one or more of the at least one processor circuit to evaluate the first hyperparameter configuration by testing an efficacy of the first hyperparameter configuration against at least one objective function of the first model.
11. The at least one of the RAM, the ROM, the flash memory, the EEPROM, the optical device, or the hard drive of claim 9 , wherein the instructions are to cause one or more of the at least one processor circuit to:
estimate a structure of the first model based on at least one or more objective functions of the first model; and
generate the surrogate model based on the structure.
12. The at least one of the RAM, the ROM, the flash memory, the EEPROM, the optical device, or the hard drive of claim 9 , wherein the instructions are to one or more of the at least one processor circuit to cause storing of a level of completion of the work request based on the first machine instance and the second machine instance.
13. The at least one of the RAM, the ROM, the flash memory, the EEPROM, the optical device, or the hard drive of claim 9 , wherein the first model is a machine learning model.
14. The at least one of the RAM, the ROM, the flash memory, the EEPROM, the optical device, or the hard drive of claim 9 , wherein the instructions are to cause one or more of the at least one processor circuit to cause the first machine instance to run the first tuning operation based on one or more tuning constraints.
15. The at least one of the RAM, the ROM, the flash memory, the EEPROM, the optical device, or the hard drive of claim 9 , wherein the instructions are to cause one or more of the at least one processor circuit to cause the transmission of the partial response after a request from the computer for a partial completion of the work request, the request for the partial completion of the work request received after the work request.
16. A method comprising:
implementing, by executing an instruction with at least one processor, a first machine instance and a second machine instance to operate in parallel to service a work request;
causing, by executing an instruction with one or more of the at least one processor, a first tuning operation to run via the first machine instance to generate a first hyperparameter configuration;
causing, by executing an instruction with one or more of the at least one processor, a second tuning operation to run via the second machine instance using Bayesian-based optimization to determine a second hyperparameter configuration;
before completion of the work request:
evaluating, by executing an instruction with one or more of the at least one processor, the first hyperparameter configuration and the second hyperparameter configuration for a first model using a surrogate model of the first model;
generating, by executing an instruction with one or more of the at least one processor, a first probability that the first hyperparameter configuration improves a performance of a computer that is to execute the first model;
generating, by executing an instruction with one or more of the at least one processor, a second probability that the second hyperparameter configuration improves the performance of the computer that is to execute the first model; and
generating, by executing an instruction with one or more of the at least one processor, a ranking of the first hyperparameter configuration and the second hyperparameter configuration based on the first probability and the second probability;
selecting, by executing an instruction with the at least one processor, the first hyperparameter configuration and the second hyperparameter configuration based on their ranking for the first model; and
causing, by executing an instruction with one or more of the at least one processor, transmission of a partial response to the computer, the partial response to include:
the ranking of the first hyperparameter configuration and the second hyperparameter configuration;
the first probability;
the second probability; and
an indication of time remaining to generate a full response based on the completion of the work request.
17. The method of claim 16 , wherein the evaluating of the first hyperparameter configuration includes testing an efficacy of the first hyperparameter configuration against at least one objective function of the first model.
18. The method of claim 16 , further including:
estimating a structure of the first model based on at least one or more objective functions of the first model; and
generating the surrogate model based on the structure.
19. The method of claim 16 , further including storing a level of completion of the work request based on the first machine instance and the second machine instance.
20. The method of claim 16 , wherein the first model is a machine learning model.
21. The method of claim 16 , wherein at least one of the first tuning operation or the second tuning operation is based on one or more tuning constraints.
22. The method of claim 16 , including transmitting the partial response after a request from the computer for a partial completion of the work request, the request for the partial completion of the work request received after the work request.