Generating machine learning model host system recommendations using multi-objective optimization
Machine learning model host system recommendations may be generated using multi-objective optimization. A machine learning model and an example payload of an inference request used to generate an inference using the machine learning model may be identified for a machine learning model host system recommendation. Multi-objective optimization may be iteratively performed that at starts from an initial set of host systems as a recommendation set of host systems for the host system recommendation, minimizes resource utilization, and maximizes inference throughput for the example payload. A Pareto front is determined for host systems in a recommendation set according to a mapping function to objective space that takes as input respective configuration parameters for different host systems being considered in a recommendation set of host systems to generate the respective inference throughput values and resource utilization values. The host system recommendation may be provided based on the Pareto front of host systems.
1 . A system, comprising:
at least one processor; and
a memory, storing program instructions that when executed by the at least one processor, cause the at least one processor to implement a machine learning model host recommendation system, configured to:
receive request for a recommendation of a provider network host system to host a machine learning model and generate an inference using the machine learning model, wherein the request for the recommendation includes an example inference request for the machine learning model;
determine an initial set of provider network host systems that are added to a recommendation set of provider network host systems;
iteratively perform multi-objective optimization that starts from the initial set of provider network host systems as a recommendation set of provider network host systems for the provider network host system recommendation, with objectives to minimize resource utilization and maximizes inference throughput for the example inference request until a termination condition is satisfied, wherein to perform the multi-objective optimization, the machine learning model host recommendation system is configured to:
based on the recommendation set of provider network host systems, identify one or more further provider network host systems to add to the recommendation set of provider network host systems according to a search technique;
generate respective inference throughput values and resource utilization values for the machine learning model to generate the inference given the example inference request at individual ones of the recommendation set of provider network host systems using a mapping function to objective space that takes as input respective configuration parameters for the individual ones of the recommendation set of provider network host systems to generate the respective inference throughput values and resource utilization values; and
determine a Pareto front of provider network host systems in the recommendation set of provider network host systems according to the respective inference throughput values and resource utilization values of the recommendation set of provider network host systems; and
select one or more host systems determined to be on the Pareto front;
return a provider network host system recommendation that includes the selected one or more host systems responsive to the request.
2 . The system of claim 1 , wherein the mapping function to objective space is a performance prediction machine learning model that predicts the respective inference throughput values and the resource utilization values for the machine learning model to generate the inference given the example inference request.
3 . The system of claim 1 , wherein to determine the initial set of provider network host systems that are added to a recommendation set of provider network host systems, the machine learning model host recommendation system is configured to apply a clustering technique that compares the machine learning model and example inference request with historical data collected for other inference requests for generating inferences using the machine learning model.
4 . The system of claim 1 , wherein the machine learning model is one of a plurality of machine learning models offered as part of a catalog of machine learning models by the provider network.
5 . A method, comprising:
identifying a machine learning model for a host system recommendation to host the machine learning model and use the machine learning model to generate an inference according to an inference request for the machine learning model;
iteratively performing multi-objective optimization that starts from an initial set of host systems as a recommendation set of host systems for the host system recommendation, with objectives to minimize resource utilization, and maximize inference throughput for the example inference request until a termination condition is satisfied, wherein performing the multi-objective optimization comprises:
based on the recommendation set of host systems, identifying one or more further host systems to add to the recommendation set of host systems according to a search technique;
generating respective inference throughput values and resource utilization values for the machine learning model to generate the inference given the example inference request at individual ones of the recommendation set of host systems using a mapping function to objective space that takes as input respective configuration parameters for the individual ones of the recommendation set of host systems to generate the respective inference throughput values and resource utilization values; and
determining a Pareto front of host systems in the recommendation set of host systems according to the respective inference throughput values and resource utilization values of the recommendation set of host systems; and
selecting one or more host systems included the Pareto front to include in the host system recommendation; and
providing the host system recommendation including the one or more selected host systems.
6 . The method of claim 5 , wherein the mapping function to objective space is a performance prediction machine learning model that predicts the respective inference throughput values and the resource utilization values for the machine learning model to generate the inference given the example inference request.
7 . The method of claim 6 , further comprising updating the performance prediction machine learning model that predicts the respective inference throughput values and the resource utilization values using a continuous learning technique based, at least in part, on recommended host system performance data.
8 . The method of claim 5 , wherein identifying the one or more further host systems to add to the recommendation set of host systems according to the search technique comprises applying differential evolution as the search technique, wherein applying differential evolution comprises scaling differences between different ones of the recommendation set of hosts to identify the one or more further host systems.
9 . The method of claim 5 , wherein iteratively performing multi-objective optimization that starts from an initial set of host systems as a recommendation set of host systems for the host system recommendation further comprises applying one or more host system constraints to exclude at least one host system from the host system recommendation set.
10 . The method of claim 5 , wherein the one or more host system constraints are specified in a request to generate the host system recommendation.
11 . The method of claim 5 , further comprising applying a clustering technique that compares the machine learning model and example inference request with historical data collected for other inference requests for generating inferences using the machine learning model to determine the initial set of host systems.
12 . The method of claim 5 , wherein providing the host system recommendation comprises generating a visualization of the Pareto front to include as part of the host system recommendation.
13 . The method of claim 5 , wherein the termination condition is specified in a request to generate the host system recommendation.
14 . One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices cause the one or more computing devices to implement:
receiving a host system recommendation request to host a machine learning model and generate an inference using the machine learning model, wherein the request includes an example inference request for the machine learning model;
iteratively performing multi-objective optimization that starts from an initial set of host systems as a recommendation set of host systems for the host system recommendation, with objectives to minimize resource utilization and maximize inference throughput for the example inference request until a termination condition is satisfied, wherein, in performing the multi-objective optimization, the program instructions cause the one or more computing devices to implement:
determining an initial set of host systems that are added to a recommendation set of host systems;
based on the recommendation set of host systems, identifying one or more further host systems to add to the recommendation set of host systems according to a search technique;
generating respective inference throughput values and resource utilization values for the machine learning model to generate the inference given the example inference request at individual ones of the recommendation set of host systems using a mapping function to objective space that takes as input respective configuration parameters for the individual ones of the recommendation set of host systems to generate the respective inference throughput values and resource utilization values; and
determining a Pareto front of host systems in the recommendation set of host systems according to the respective inference throughput values and resource utilization values of the recommendation set of host systems; and
selecting one or more host systems included the Pareto front to include in the host system recommendation; and
providing the host system recommendation including the one or more selected host systems responsive to the request.
15 . The one or more non-transitory, computer-readable storage media of claim 14 , wherein the mapping function to objective space is a performance prediction machine learning model that predicts the respective inference throughput values and the resource utilization values for the machine learning model to generate the inference given the example inference request.
16 . The one or more non-transitory, computer-readable storage media of claim 14 , storing further program instructions that when executed by the one or more computing devices cause the one or more computing devices to further implement applying one or more host system constraints to exclude at least one host system from the host system recommendation set.
17 . The one or more non-transitory, computer-readable storage media of claim 16 , wherein the one or more host system constraints are based on availability of the host system for a user account of a provider network associated with a request to generate the host system recommendation.
18 . The one or more non-transitory, computer-readable storage media of claim 14 , storing further program instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to further implement apply a clustering technique that compares the machine learning model and example inference request with historical data collected for other inference requests for generating inferences using the machine learning model to determine the initial set of host systems.
19 . The one or more non-transitory, computer-readable storage media of claim 14 , wherein in wherein providing the host system recommendation, the program instructions cause the one or more computing devices to further implement generating a box plot visualization of different host systems to include as part of the host system recommendation.
20 . The one or more non-transitory, computer-readable storage media of claim 14 , storing further program instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to further implement:
receiving a different host system recommendation request for a different machine learning model to generate a different inference given a different example inference request for the different machine learning model;
applying a clustering technique that compares the different machine learning model and example inference request with historical data collected for other inference requests for generating inferences using the different machine learning model to determine one or more host systems to recommend for the different example inference request for the different machine learning model.