Dynamic selection from among multiple candidate generative models with differing computational efficiencies
Implementations disclose selecting, in response to receiving a request and from among multiple candidate generative models (e.g., multiple candidate large language models (LLMs)) with differing computational efficiencies, a particular generative model to utilize in generating a response to the request. Those implementations reduce latency and/or conserve computational resource(s) through selection, for various requests, of a more computationally efficient generative model for utilization in lieu of a less computationally efficient generative model. Further, those implementations seek to achieve such benefits, through utilization of more computationally efficient generative models, while also still selectively utilizing less computationally efficient generative models for certain requests to mitigate occurrences of a generated response being inaccurate and/or under-specified. This, in turn, can mitigate occurrences of computational and/or network inefficiencies that result from a user issuing a follow-up request to cure the inaccuracies and/or under-specification of a generated response.
1 . A method implemented by one or more processors, the method comprising:
receiving a request, the request including a query formulated based on user interface input at a client device;
processing, using a trained machine learning model, request features of the request to generate output;
selecting, based on the output generated using the trained machine learning model and from a plurality of candidate generative models, a particular generative model to utilize in responding to the request,
wherein the candidate generative models include the particular generative model and an additional generative model that is less computationally efficient than the particular generative model, and
wherein the trained machine learning model is not among the candidate generative models, is more computationally efficient than the particular generative model, and is more computationally efficient than the additional generative model;
in response to selecting the particular generative model based on the output generated using the trained machine learning model:
causing the particular generative model to be utilized, without any utilization of any other of the candidate generative models, in generating a response to the request; and
causing the response, generated utilizing the particular generative model, to be rendered in response to receiving the query.
2 . The method of claim 1 , wherein the output indicates, for each of the candidate generative models, a corresponding probability of generating a correct response, the corresponding probabilities including a particular probability for the particular generative model and an additional probability for the additional generative model; and
wherein selecting the particular generative model is based on the particular probability, for the particular generative model, indicated by the output.
3 . The method of claim 2 , wherein selecting the particular generative model is further based on the additional probability, for the additional generative model, indicated by the output.
4 . The method of claim 3 , wherein the additional probability indicates a greater probability of generating a correct response than does the particular probability.
5 . The method of claim 3 , wherein selecting the particular generative model comprises selecting the particular generative model based on:
determining the particular probability satisfies a first absolute threshold,
determining the particular probability satisfies a relative threshold relative to the additional probability, and/or
determining the additional probability fails to satisfy a second absolute threshold, the second absolute threshold being more restrictive than the first absolute threshold.
6 . The method of claim 5 , further comprising:
determining a current server load that characterizes a magnitude of current computational resource utilization in processing requests; and
determining the first absolute threshold, the relative threshold, and/or the second absolute threshold as a function of the current server load.
7 . The method of claim 1 , wherein the request features include one or more query features that are determined based on the query.
8 . The method of claim 7 , wherein the query includes text and wherein the query features include one or more features generated based on the text.
9 . The method of claim 1 , wherein the request is part of a multi-turn dialog and wherein the request features include dialog context features that are based on one or more prior requests of the multi-turn dialog and/or based on one or more prior responses of the multi-turn dialog.
10 . The method of claim 1 , wherein the request further includes an attribute identifier associated with the client device and/or with a user account of a user that provided the user interface input, and wherein the request features include an attribute feature that is based on the attribute identifier.
11 . The method of claim 1 , wherein the additional generative model includes a first quantity of parameters, the particular generative model includes a second quantity of parameters, and the second quantity is at least twenty-five percent less than the first quantity.
12 . The method of claim 11 , wherein the second quantity is at least thirty-five percent less than the first quantity.
13 . The method of claim 12 , wherein the second quantity is at least fifty percent less than the first quantity.
14 . The method of claim 1 , further comprising:
determining a current server load that characterizes a magnitude of current computational resource utilization in processing requests;
wherein selecting, from the plurality of candidate generative models, the particular generative model, is further based on the current server load.
15 . The method of claim 1 , wherein the candidate generative models further include a third generative model.
16 . The method of claim 1 , further comprising:
receiving an additional request, the additional request including an additional query formulated based on additional user interface input at the client device;
processing, using the trained machine learning model, additional request features of the additional request to generate additional output;
selecting, based on the additional output generated using the trained machine learning model and from the plurality of candidate generative models, the additional generative model for utilization in responding to the additional request,
in response to selecting the additional generative model:
causing the additional generative model to be utilized, without any utilization of the more computationally efficient particular generative model, in generating an additional response to the additional request; and
causing the additional response, generated utilizing the additional generative model, to be rendered in response to receiving the additional query.
17 . The method claim 1 , further comprising:
generating, based on the request and using the particular generative model, the response to the request.
18 . A system, comprising:
memory storing instructions;
one or more processors operable to execute the instructions to:
receive a request, the request including a query formulated based on user interface input at a client device;
process, using a trained machine learning model, request features of the request to generate output;
select, based on the output generated using the trained machine learning model and from a plurality of candidate generative models, a particular generative model to utilize in responding to the request,
wherein the candidate generative models include the particular generative model and an additional generative model that is less computationally efficient than the particular generative model, and
wherein the trained machine learning model is not among the candidate generative models, is more computationally efficient than the particular generative model, and is more computationally efficient than the additional generative model;
in response to selecting the particular generative model based on the output generated using the trained machine learning model:
cause the particular generative model to be utilized, without any utilization of any other of the candidate generative models, in generating a response to the request; and
cause the response, generated utilizing the particular generative model, to be rendered in response to receiving the query.
19 . The system of claim 18 , wherein one or more of the processors are further operable to execute the instructions to:
determine a current server load that characterizes a magnitude of current computational resource utilization in processing requests;
wherein in selecting, from the plurality of candidate generative models, the particular generative model, one or more of the processors are to select the particular generative model further based on the current server load.