HYBRID LANGUAGE MODEL ARCHITECTURE FOR API ORCHESTRATION INCLUDING CHAIN OF THOUGHT
A hybrid architecture that combines the efficiency of small language models with the accuracy of large language models for enhanced selection of requested functionality and identification of data visualizations using a network system is described. For instance, an orchestration module employs a hybrid architecture using both small language models and large language models to generate API payloads for clients in a way that harnesses the benefits of both models.
1 . A method for generating a payload response for a requested functionality in a system environment, the method comprising:
receiving, at a network system, a payload request from a client device, the payload request comprising a natural language request for the requested functionality;
applying, at the network system, a small language model to the payload request to determine a set of candidate APIs from a catalog based on embeddings generated by the small language model that map the natural language in the payload request to the functionality of the APIs in the catalog;
applying, at the network system, a large language model to the payload request and the candidate APIs to determine a selected API that performs the requested functionality, the large language model interpreting context and intent of the payload request to select a candidate API from the set of candidate APIs as the selected API based on the context and intent;
executing the selected API to generate a payload response including the requested functionality; and
transmitting the payload response to the client device.
2 . The method of claim 1 , further comprising:
generating the catalog for the network system by:
accessing a plurality of APIs within a system environment comprising the network system;
determining metadata associated with a functionality of each of the APIs in the plurality of APIs; and
wherein the set of candidate APIs are selected from the plurality of APIs.
3 . The method of claim 2 , further comprising:
training the small language model to identify candidate APIs by mapping metadata describing the functionality of each API to language samples representing a plurality of payload requests.
4 . The method of claim 2 , wherein accessing a plurality of APIs within the system environment comprises accessing APIs from one or more additional network systems.
5 . The method of claim 1 , wherein applying the large language model to the payload request and the set candidate APIs further comprises:
providing metadata associated with each candidate API to the large language model, and
wherein selecting the candidate API as the selected API is further based on the metadata.
6 . The method of claim 1 , wherein selecting candidate APIs further comprises:
determining, for each API of a plurality of APIs in the catalog, a score quantifying a likelihood the API is semantically or syntactically relevant to language in the payload request;
selecting a number of APIs having a highest score as the candidate APIs.
7 . The method of claim 1 , wherein selecting candidate APIs further comprises:
determining, for each API of a plurality of APIs in the catalog, a score quantifying a likelihood the API is semantically or syntactically relevant to language in the payload request;
selecting the APIs having scores above a threshold score as the candidate APIs.
8 . The method of claim 1 , wherein the network system executes the selected API and transmits the payload response to the client device.
9 . The method of claim 1 , further comprising:
providing a location of the selected API to the client device; and
wherein a system hosting the location executes the selected API and the system transmits the payload response to the client device.
10 . The method of claim 1 , wherein applying the large language model to the payload request and the candidate APIs to determine the selected API that performs the requested functionality comprises:
selecting one or more additional APIs for an API chain, and wherein:
each of the one or more additional APIs provide a partial functionality related to the requested functionality, and
the API chain, in aggregate, provides the requested functionality.
11 . A non-transitory computer-readable storage medium comprising computer program instructions for a payload response for a requested functionality in a system environment, the computer program instructions, when executed, causing the one or more processors to:
receive, at a network system, a payload request from a client device, the payload request comprising a natural language request for the requested functionality;
apply, at the network system, a small language model to the payload request to determine a set of candidate APIs from a catalog based on embeddings generated by the small language model that map the natural language in the payload request to the functionality of the APIs in the catalog;
apply, at the network system, a large language model to the payload request and the candidate APIs to determine a selected API that performs the requested functionality, the large language model interpreting context and intent of the payload request to select a candidate API from the set of candidate APIs as the selected API based on the context and intent;
execute the selected API to generate a payload response including the requested functionality; and
transmit the payload response to the client device.
12 . The non-transitory computer-readable storage medium of claim 11 , wherein the computer program instructions, when executed, cause the one or more processors to:
generate the catalog for the network system by:
accessing a plurality of APIs within a system environment comprising the network system;
determining metadata associated with a functionality of each of the APIs in the plurality of APIs; and
wherein the set of candidate APIs are selected from the plurality of APIs.
13 . The non-transitory computer-readable storage medium of claim 12 , further comprising:
train the small language model to identify candidate APIs by mapping metadata describing the functionality of each API to language samples representing a plurality of payload requests.
14 . The non-transitory computer-readable storage medium of claim 12 , wherein accessing a plurality of APIs within the system environment causes the one or more processors to:
access APIs from one or more additional network systems.
15 . The non-transitory computer-readable storage medium of claim 11 , wherein applying the large language model to the payload request and the set candidate APIs causes the one or more processors to:
provide metadata associated with each candidate API to the large language model, and
wherein selecting the candidate API as the selected API is further based on the metadata.
16 . The non-transitory computer-readable storage medium of claim 11 , wherein selecting candidate APIs causes the one or more processors to:
determine, for each API of a plurality of APIs in the catalog, a score quantifying a likelihood the API is semantically or syntactically relevant to language in the payload request;
select a number of APIs having a highest score as the candidate APIs.
17 . The non-transitory computer-readable storage medium of claim 11 , wherein selecting candidate APIs causes the one or more processors to:
determine, for each API of a plurality of APIs in the catalog, a score quantifying a likelihood the API is semantically or syntactically relevant to language in the payload request;
select the APIs having scores above a threshold score as the candidate APIs.
18 . The non-transitory computer-readable storage medium of claim 11 , wherein the network system executes the selected API and transmits the payload response to the client device.
19 . The non-transitory computer-readable storage medium of claim 11 , wherein the computer program instructions, when executed, cause the one or more processors to:
provide a location of the selected API to the client device; and
wherein a system hosting the location executes the selected API and the system transmits the payload response to the client device.
20 . The non-transitory computer-readable storage medium of claim 11 , wherein applying the large language model to the payload request and the candidate APIs to determine the selected API that performs the requested functionality comprises:
select one or more additional APIs for an API chain, and wherein:
each of the one or more additional APIs provide a partial functionality related to the requested functionality, and
the API chain, in aggregate, provides the requested functionality.