SYSTEMS AND METHODS FOR AN ACCELERATED TUNING OF HYPERPARAMETERS OF A MODEL USING A MACHINE LEARNING-BASED TUNING SERVICE
A system and method for accelerated tuning of hyperparameters includes receiving a multi-task tuning work request for tuning hyperparameters of a model, wherein the multi-task tuning work request includes: a full tuning task for tuning hyperparameters, wherein the full tuning task includes a first set of tuning parameters governing a first tuning operation; a partial tuning task for tuning the hyperparameters of the model, wherein the partial tuning task includes a second distinct set of tuning parameters governing a second tuning operation; executing the first tuning operation and the second tuning operation; generating a first suggestion set and a second suggestion set of one or more proposed values for the hyperparameters based on the execution of the full tuning task and the partial tuning task; and setting the partial tuning task as a proxy for the full tuning task thereby accelerating a tuning of the hyperparameters of the model.
1 . A system to accelerate tuning of hyperparameters, the system comprising:
memory;
programmable circuitry; and
instructions in the memory to cause the programmable circuitry to:
access a multi-task tuning work request to tune hyperparameters of a model of a subscriber to a tuning service, the multi-task tuning work request to include:
(i) a full tuning task to tune the hyperparameters of the model, the full tuning task to include first tuning parameters governing a first tuning operation of the tuning service; and
(ii) a partial tuning task to tune the hyperparameters of the model, the partial tuning task to include second tuning parameters governing a second tuning operation of the tuning service;
during the first tuning operation, provide as input to tune the model a corpus of training data based on one or more of the first tuning parameters of the full tuning task;
execute the first tuning operation of the full tuning task based on the first tuning parameters;
during the second tuning operation, sample a subset of the corpus of training data as input to tune the model based on one or more of the second tuning parameters of the partial tuning task;
execute the second tuning operation of the partial tuning task based on the second tuning parameters;
generate a first suggestion set, the first suggestion set to include one or more first proposed values for the hyperparameters based on the execution of the full tuning task; and
generate a second suggestion set, the second suggestion set to include one or more second proposed values for the hyperparameters based on the execution of the partial tuning task,
wherein after an identified performance metric of the model using the one or more second proposed values for the hyperparameters corresponding to the partial tuning task satisfies a performance threshold, set the partial tuning task as a proxy for the full tuning task to accelerate tuning of the hyperparameters of the model.