DYNAMICALLY SELECTING ARTIFICIAL INTELLIGENCE MODELS AND HARDWARE ENVIRONMENTS TO EXECUTE TASKS
The present disclosure relates to systems, non-transitory computer-readable media, and methods for selecting machine-learning models and hardware environments for executing a task. In particular, in one or more embodiments, the disclosed systems select a designated machine-learning model for executing a task based on workload features of the task and task routing metrics for a plurality of machine-learning models. In addition, in one or more embodiments, the disclosed systems select a designated hardware environment for executing the task based on workload features for the task and task routing metrics for a plurality of hardware environments. In some embodiments, the disclosed systems select a fallback machine-learning model and a fallback hardware environment for executing the task if the designated machine-learning model or designated hardware environment are unavailable. Moreover, in one or more embodiments, the disclosed systems can pause and initiate tasks based on bandwidth availability.
1 . A computer-implemented method comprising:
receiving, from a device connected by a network, workload data requesting execution of a task using a machine-learning model;
extracting, from the workload data, workload features defining characteristics of the task;
generating a ranked list of potential machine-learning models for executing the task based on the workload features defining the characteristics of the task;
selecting, based on the ranked list of potential machine-learning models, a primary machine-learning model for executing the task; and
selecting, based on the ranked list of potential machine-learning models, a fallback machine-learning model for executing the task if the primary machine-learning model is unavailable.
2 . The computer-implemented method of claim 1 , further comprising extracting workload features by determining an estimated processing requirement or an estimated storage requirement for executing the task.
3 . The computer-implemented method of claim 1 , wherein generating the ranked list of potential machine-learning models to execute the task comprises providing the workload features to a model selection machine-learning model, the model selection machine-learning model trained to generate ranked lists of potential machine-learning models based on workload features of tasks and characteristics associated with each of a plurality of machine learning models.
4 . The computer-implemented method of claim 1 , further comprising:
determining a model state for each of a plurality of machine-learning models; and
wherein generating the ranked list of potential machine-learning models for executing the task comprises providing the workload features and the model state for each of the plurality of machine-learning models to a model selection machine-learning model.
5 . The computer-implemented method of claim 1 , further comprising determining to route the task to the fallback machine-learning model based on determining an updated modal state corresponding to the primary machine-learning model.
6 . The computer-implemented method of claim 1 , further comprising:
receiving second workload data requesting execution of a second task;
extracting, from the second workload data, second workload features defining second task characteristics; and
generating a second ranked list of potential machine-learning models for executing the second task based on the second workload features, wherein the second ranked list of potential machine-learning models is different from the ranked list of potential machine-learning models.
7 . The computer-implemented method of claim 1 , wherein the ranked list of potential machine-learning models comprises at least one local machine-learning model and at least one third-party machine-learning model.
8 . A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computer system to:
receive, from a device connected by a network, workload data requesting execution of a task;
generate a ranked list of potential machine-learning models for executing the task based on workload features extracted from the workload data corresponding to the task;
select, based on the ranked list of potential machine-learning models, a primary machine-learning model for executing the task; and
select, based on the ranked list of potential machine-learning models, a fallback machine-learning model for executing the task if the primary machine-learning model is unavailable.
9 . The non-transitory computer-readable medium of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer system to extract workload features by determining an estimated processing requirement or an estimated storage requirement for executing the task.
10 . The non-transitory computer-readable medium of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
determine one or more capability metrics for each of a plurality of machine-learning models; and
wherein generating the ranked list of potential machine-learning models for executing the task is further based on the one or more capability metrics for each of the plurality of machine-learning models.
11 . The non-transitory computer-readable medium of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer system to route the task to the fallback machine-learning model based on determining an updated capability metric corresponding to the primary machine-learning model.
12 . The non-transitory computer-readable medium of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
receive additional workload data requesting execution of an additional task; and
generate an additional ranked list of potential machine-learning models for executing the additional task based on additional workload features extracted from the additional workload data, wherein the additional ranked list of potential machine-learning models is different from the ranked list of potential machine-learning models.
13 . The non-transitory computer-readable medium of claim 8 , wherein the ranked list of potential machine-learning models comprises at least one local machine-learning model and at least one third-party machine-learning model.
14 . The non-transitory computer-readable medium of claim 8 , wherein generating the ranked list of potential machine-learning models to execute the task comprises providing the workload features to a model selection machine-learning model.
15 . A system comprising:
at least one processor; and
at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
receive workload data requesting execution of a task;
generate a ranked list of potential machine-learning models for executing the task based on workload features extracted from the workload data corresponding to the task;
based on the ranked list of potential machine-learning models, select:
a primary machine-learning model for executing the task; and
a fallback machine-learning model for executing the task if the primary machine-learning model is unavailable; and
route the task to the primary machine-learning model for execution of the task.
16 . The system of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the system to:
determine an updated model state for the primary machine-learning model; and
re-route the task to the fallback machine-learning model based on determining the updated model state for the primary machine-learning model.
17 . The system of claim 15 , wherein generating a ranked list of potential machine-learning models for executing the task based on workload features comprises aligning task requirements with model capabilities of a plurality of machine-learning models based on one or more of the workload features for the task and one or more capability metrics associated with each of the plurality of machine-learning models.
18 . The system of claim 15 , wherein generating the ranked list of potential machine-learning models for executing the task comprises providing the workload features a model selection machine-learning model.
19 . The system of claim 18 , further comprising instructions that, when executed by the at least one processor, cause the system to:
determine execution performance based on executing the task with the primary machine-learning model; and
update parameters of the model selection machine-learning model based on the execution performance.
20 . The system of claim 15 , wherein the ranked list of potential machine-learning models comprises at least one local machine-learning model and at least one third-party machine-learning model.