Dynamically selecting artificial intelligence models and hardware environments to execute tasks
The present disclosure relates to systems, non-transitory computer-readable media, and methods for selecting machine-learning models and hardware environments for executing a task. In particular, in one or more embodiments, the disclosed systems select a designated machine-learning model for executing a task based on workload features of the task and task routing metrics for a plurality of machine-learning models. In addition, in one or more embodiments, the disclosed systems select a designated hardware environment for executing the task based on workload features for the task and task routing metrics for a plurality of hardware environments. In some embodiments, the disclosed systems select a fallback machine-learning model and a fallback hardware environment for executing the task if the designated machine-learning model or designated hardware environment are unavailable. Moreover, in one or more embodiments, the disclosed systems can pause and initiate tasks based on bandwidth availability.
1. A computer-implemented method comprising:
receiving, from a device connected by a network to a content management system, workload data requesting execution of a task using a machine-learning model;
extracting, from the workload data, workload features defining characteristics of the task;
determining task routing metrics for a plurality of hardware environments hosted in respective network environments and a plurality of machine-learning models in respective network environments;
determining a historical quality metric indicating how one or more machine learning models of the plurality of machine-learning models will execute the task based on one or more user feedback metrics; and
selecting, based on an output of a model selection machine-learning model that utilizes the historical quality metric, from the plurality of hardware environments and from the plurality of machine-learning models, a designated hardware environment and a designated machine-learning model for executing the task based on the workload features and the task routing metrics.
2. The computer-implemented method of claim 1 , wherein determining task routing metrics for the plurality of hardware environments further comprises:
determining a hardware state for each hardware environment of the plurality of hardware environments; and
wherein selecting, from the plurality of hardware environments, the designated hardware environment for executing the task is based in part on the hardware state of the designated hardware environment.
3. The computer-implemented method of claim 1 , wherein determining task routing metrics for the plurality of hardware environments further comprises:
determining, based on the workload features, a financial cost metric, an execution time metric, an execution cost metric, or a model fit metric for executing the task on each hardware environment of the plurality of hardware environments; and
wherein selecting, from the plurality of hardware environments, the designated hardware environment for executing the task is based on two or more of the financial cost metric, the execution time metric, the execution cost metric, and the model fit metric.
4. The computer-implemented method of claim 1 , wherein selecting the designated hardware environment further comprises:
analyzing the workload data and the task routing metrics for the plurality of hardware environments to determine an optimal machine-learning model for executing the task from the plurality of machine-learning models; and
selecting the designated machine-learning model for executing the task based on determining that the designated machine-learning model is the optimal machine-learning model from the plurality of machine-learning models.
5. The computer-implemented method of claim 4 , wherein determining that the designated machine-learning model is the optimal machine-learning model further comprises:
generating, utilizing a smart pocket machine-learning model, an optimization metric; and
determining that the designated machine-learning model is the optimal machine-learning model based on the optimization metric.
6. The computer-implemented method of claim 1 , wherein extracting workload features defining characteristics of the task further comprises determining an estimated processing requirement and an estimated storage requirement for executing the task.
7. The computer-implemented method of claim 1 , wherein selecting the designated hardware environment further comprises:
analyzing the workload data and the task routing metrics for the plurality of hardware environments of the plurality of hardware environments, wherein the plurality of hardware environments comprises one or more hardware environments and one or more third-party hardware environments; and
selecting the designated hardware environment for executing the task based on analyzing the workload data and the task routing metrics.
8. The computer-implemented method of claim 1 , wherein selecting the designated hardware environment further comprises:
performing probabilistic load balancing of each hardware environments of the plurality of hardware environments for executing the task; and
selecting the designated hardware environment based on the probabilistic load balancing.
9. The computer-implemented method of claim 1 , wherein selecting the designated hardware environment further comprises:
accessing the one or more user feedback metrics about executing tasks using one or more machine-learning models of the plurality of machine-learning models;
determining the historical quality metric indicating how the one or more machine-learning models of the plurality of machine-learning models will execute the task based on the one or more user feedback metrics; and
selecting the designated machine-learning model for executing the task based on the historical quality metric.
10. The computer-implemented method of claim 1 , wherein selecting the designated hardware environment further comprises:
identifying a capability or specialty associated with one or more machine-learning models of the plurality of machine-learning models; and
selecting the designated machine-learning model for executing the task based on alignment of the capability or the specialty of the designated hardware environment with the task.
11. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computer system to:
receive, from a device connected by a network to a content management system, workload data requesting execution of a task using a machine-learning model;
extract, from the workload data, workload features defining characteristics of the task;
determine task routing metrics for a plurality of hardware environments hosted in respective network environments and a plurality of machine-learning models in respective network environments;
determine a historical quality metric indicating how one or more machine learning models of the plurality of machine-learning models will execute the task based on one or more user feedback metrics; and
select, based on an output of a model selection machine-learning model that utilizes the historical quality metric, from the plurality of hardware environments and from the plurality of machine-learning models and based on the workload features and the task routing metrics:
a first designated hardware environment and a first designated machine-learning model for executing a first portion of the task; and
a second designated hardware environment and a second designated machine-learning model for executing a second portion of the task.
12. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
determine a hardware state for each hardware environment of the plurality of hardware environments;
determine a model state for each machine-learning model for the plurality of machine-learning models;
select the first designated hardware environment for executing the first portion of the task is based on a first hardware state for the first designated hardware environment and a first model state for a first machine-learning model; and
select the second designated hardware environment for executing the second portion of the task is based on a second hardware state for the second designated hardware environment and a second model state for a second machine-learning model.
13. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
determine, based on the workload features, a financial cost metric, an execution time metric, an execution cost metric, or a model fit metric for executing the task on each machine-learning model of the plurality of machine-learning models;
select the first designated machine-learning model for executing the first portion of the task based on two or more of the financial cost metric, the execution time metric, the execution cost metric, or the model fit metric for executing the first portion of the task on the first designated machine-learning model; and
select the second designated machine-learning model for executing the second portion of the task based on two or more of the financial cost metric, the execution time metric, the execution cost metric, or the model fit metric for executing the second portion of the task on the second designated machine-learning model.
14. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
analyze the workload data and the task routing metrics for the plurality of hardware environments;
select the first designated hardware environment for executing the first portion of the task based on determining that the first designated hardware environment is a first optimal hardware environment from the plurality of hardware environments; and
select the second designated hardware environment for executing the second portion of the task based on determining that the second designated hardware environment is a second optimal hardware environment for executing the second portion of the task.
15. The non-transitory computer-readable medium of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
perform probabilistic load balancing of each hardware environment of the plurality of hardware environments for executing the task; and
select the first designated hardware environment for executing the first portion of the task and the second designated hardware environment for executing the second portion of the task based on the probabilistic load balancing.
16. A system comprising:
at least one processor; and
at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
receive, from a device connected by a network to a content management system, workload data requesting execution of a task using a machine-learning model;
extract, from the workload data, workload features defining characteristics of the task;
determine task routing metrics for a plurality of hardware environments hosted in respective network environments and a plurality of machine-learning models in respective network environments;
determine a historical quality metric indicating how one or more machine learning models of the plurality of machine-learning models will execute the task based on a user feedback metric; and
select, utilizing an output of a smart pocket machine-learning model that utilizes the historical quality metric, a designated hardware environment from the plurality of hardware environments and a designated machine-learning model from the plurality of machine-learning models for executing the task based on the workload features and the task routing metrics.
17. The system of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the system to:
utilize the smart pocket machine-learning model to compare each hardware environment of the plurality of hardware environments and each machine learning model of the plurality of machine-learning models based on the workload features and the task routing metrics; and
select the designated hardware environment from the plurality of hardware environments and the designated machine-learning model from the plurality of machine-learning models based on an output of the smart pocket machine-learning model.
18. The system of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the system to:
receive additional task routing metrics for an additional hardware environment; and
update parameters of the smart pocket machine-learning model based on the additional task routing metrics of the additional hardware environment.
19. The system of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the system to select the designated hardware environment by:
generate, utilizing the smart pocket machine-learning model, an optimization metric for each hardware environment of the plurality of hardware environments; and
select the designated hardware environment based on the optimization metric.
20. The system of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the system to:
receive user feedback data indicating a user satisfaction with execution of the task; and
update parameters of the smart pocket machine-learning model based on the user feedback data.