Dynamically selecting artificial intelligence models and hardware environments to execute tasks
The present disclosure relates to systems, non-transitory computer-readable media, and methods for selecting machine-learning models and hardware environments for executing a task. In particular, in one or more embodiments, the disclosed systems select a designated machine-learning model for executing a task based on workload features of the task and task routing metrics for a plurality of machine-learning models. In addition, in one or more embodiments, the disclosed systems select a designated hardware environment for executing the task based on workload features for the task and task routing metrics for a plurality of hardware environments. In some embodiments, the disclosed systems select a fallback machine-learning model and a fallback hardware environment for executing the task if the designated machine-learning model or designated hardware environment are unavailable. Moreover, in one or more embodiments, the disclosed systems can pause and initiate tasks based on bandwidth availability.
1 . A computer-implemented method comprising:
receiving, from a device connected by a network to a content management system, workload data requesting execution of a task using a machine-learning model;
extracting, from the workload data, workload features defining estimated computational requirements for executing the task;
determining a historical quality metric for each machine-learning model of a plurality of machine-learning models based on historical user feedback data about executing tasks using one or more machine-learning models of the plurality of machine-learning models;
selecting a primary machine-learning model for executing the task and a fallback machine-learning model for executing the task if the primary machine-learning model is unavailable based on the workload features defining the estimated computational requirements for executing the task and a first historical quality metric for the primary machine-learning model and a second historical quality metric for the fallback machine-learning model; and
based on detecting that the primary machine-learning model is unavailable prior to executing the task, providing the workload data to a computing environment of the fallback machine-learning model for executing the task.
2 . The computer-implemented method of claim 1 , wherein extracting the workload features defining estimated computational requirements for executing the task further comprises determining an estimated processing requirement and an estimated storage requirement for executing the task.
3 . The computer-implemented method of claim 1 , wherein selecting the primary machine-learning model and the fallback machine-learning model further comprises:
generating optimization metrics for each machine-learning model of a plurality of machine-learning models; and
selecting the primary machine-learning model and the fallback machine-learning model based on the optimization metrics.
4 . The computer-implemented method of claim 1 , wherein selecting the primary machine-learning model and the fallback machine-learning model further comprises:
determining a model state for a plurality of machine-learning models;
selecting the primary machine-learning model based in part on a first model state of the primary machine-learning model; and
selecting the fallback machine-learning model based in part on a second model state of the fallback machine-learning model.
5 . The computer-implemented method of claim 1 , wherein selecting the primary machine-learning model and the fallback machine-learning model further comprises:
determining, based on the workload features, a financial cost metric, an execution time metric, an execution cost metric, or a model fit metric for executing the task on each machine-learning models in a plurality of machine-learning models; and
selecting the primary machine-learning model and the fallback machine-learning model is based on two or more of the financial cost metric, the execution time metric, the execution cost metric, or the model fit metric.
6 . The computer-implemented method of claim 1 , further comprising:
selecting an additional machine-learning model for executing the task based on the workload features; and
based on detecting that the fallback machine-learning model is unavailable, providing the workload data to a computing environment of the additional machine-learning model for executing the task.
7 . The computer-implemented method of claim 1 , further comprising selecting a trained machine-learning model as the primary machine-learning model and a third-party trained machine-learning model as the fallback machine-learning model.
8 . The computer-implemented method of claim 1 , further comprising:
detecting that the primary machine-learning model is unavailable based on identifying that a first hardware environment associated with the primary machine-learning model is unavailable to execute the task; and
providing the workload data to a second hardware environment associated with the fallback machine-learning model based on identifying that the second hardware environment is available to execute the task.
9 . The computer-implemented method of claim 1 , wherein selecting the primary machine-learning model and the fallback machine-learning model further comprises:
performing a software domain analysis of each machine-learning model of a plurality of machine-learning models for executing the task; and
selecting the primary machine-learning model and the fallback machine-learning model based on the software domain analysis.
10 . A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computer system to:
receive, from a device connected by a network to a content management system, workload data requesting execution of a task using a machine-learning model;
extract, from the workload data, workload features defining estimated computational requirements for executing the task;
determine task routing metrics for a plurality of machine-learning models hosted in respective network environments;
determine a historical quality metric for each machine-learning model of a plurality of machine-learning models based on historical user feedback data about executing tasks using one or more machine-learning models of the plurality of machine-learning models;
select a primary machine-learning model for executing the task and a fallback machine-learning model from the plurality of machine-learning models for executing the task if the primary machine-learning model is unavailable based on the workload features defining the estimated computational requirements for executing the task and a first historical quality metric for the primary machine-learning model and a second historical quality metric for the fallback machine-learning model; and
based on detecting that the primary machine-learning model is unavailable prior to executing the task, provide the workload data to a computing environment of the fallback machine-learning model for executing the task.
11 . The non-transitory computer-readable medium of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the computer system to select the primary machine-learning model and the fallback machine-learning model by:
determining task routing metrics for the plurality of machine-learning models by determining a financial cost metric, an execution time metric, an execution cost metric, or a model fit metric for executing the task on each machine-learning models of the plurality of machine-learning models; and
selecting, from the plurality of machine-learning models, the primary machine-learning model and the fallback machine-learning model based on two or more of the financial cost metric, the execution time metric, the execution cost metric, or the model fit metric for executing the task.
12 . The non-transitory computer-readable medium of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the computer system to select the primary machine-learning model and the fallback machine-learning model by:
generating, based on the workload data and the task routing metrics, an optimization metric for each machine-learning model of the plurality of machine-learning models; and
selecting the primary machine-learning model and the fallback machine-learning model from the plurality of machine-learning models based on a first optimization metric for the primary machine-learning model and a second optimization metric for the fallback machine-learning model.
13 . The non-transitory computer-readable medium of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
select the primary machine-learning model and the fallback machine-learning model by utilizing a model selection machine-learning model to select the primary machine-learning model and the fallback machine-learning model;
receive user feedback data indicating a user satisfaction with the fallback machine-learning model executing the task; and
update parameters of the model selection machine-learning model based on the user feedback data.
14 . The non-transitory computer-readable medium of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the computer system to select the primary machine-learning model and the fallback machine-learning model by:
identifying a capability or a specialty associated with one or more machine-learning models of the plurality of machine-learning models; and
selecting the primary machine-learning model for executing the task based on alignment of the capability or the specialty of the primary machine-learning model with the task.
15 . The non-transitory computer-readable medium of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the computer system to select a trained machine-learning model as the primary machine-learning model and a third-party trained machine-learning model as the fallback machine-learning model.
16 . A system comprising:
at least one processor; and
at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
receive, from a device connected by a network to a content management system, workload data requesting execution of a task using a machine-learning model;
extract, from the workload data, workload features defining estimated computational requirements for executing the task;
determine a historical quality metric for each machine-learning model of a plurality of machine-learning models based on historical user feedback data about executing tasks using one or more machine-learning models of the plurality of machine-learning models;
select, utilizing a model selection machine-learning model, a primary machine-learning model for executing the task and a fallback machine-learning model for executing the task if the primary machine-learning model is unavailable based on the workload features defining the estimated computational requirements for executing the task and a first historical quality metric for the primary machine-learning model and a second historical quality metric for the fallback machine-learning model; and
based on detecting that the primary machine-learning model is unavailable prior to executing the task, provide the workload data to a computing environment of the fallback machine-learning model for executing the task.
17 . The system of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the system to:
utilize the model selection machine-learning model to compare each machine-learning model in a plurality of machine-learning models based on the workload features, where the plurality of machine-learning models comprises one or more trained models and one or more third-party trained models; and
select the primary machine-learning model and the fallback machine-learning model from the plurality of machine-learning models based on an output of the model selection machine-learning model.
18 . The system of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the system to:
train the model selection machine-learning model based on task routing metrics of a plurality of machine-learning models, where the plurality of machine-learning models comprises the primary machine-learning model and the fallback machine-learning model;
receive additional task routing metrics for an additional machine-learning model; and
update parameters of the model selection machine-learning model based on the additional task routing metrics of the additional machine-learning model.
19 . The system of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the system to:
receive user feedback data indicating a user satisfaction with performance of the primary machine-learning model; and
update parameters of the model selection machine-learning model based on the user feedback data.
20 . The system of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the system to:
generate, utilizing the model selection machine-learning model, an optimization metric for each machine-learning model of a plurality of machine-learning models; and
select the primary machine-learning model and the fallback machine-learning model from the plurality of machine-learning models based on a first optimization metric for the primary machine-learning model and a second optimization metric for the fallback machine-learning model.