Framework for language model copilot development
The present disclosure relates to systems and methods for creating a copilot. The copilot uses plugins to provide additional features and functionalities to the copilot. The systems and methods use a large language model (LLM) pipeline to generate a knowledge resource used by the plugins and/or an LLM in the copilot to answer queries from a user.
1 . A method comprising:
receiving, at a large language model (LLM), a query and a description of a plurality of tools available for use with the LLM;
determining, by the large language model, to use a tool of the plurality of tools in response to a description of the tool of the plurality of tools matching a portion of the query;
providing a call to the tool with the query;
receiving information from the tool in response to the call;
using, by the LLM, the information in providing a response to the query; and
outputting the response to the query.
2 . The method of claim 1 , wherein each tool of the plurality of tools performs a function that provides the LLM additional capabilities in providing a response to the query and the description of the plurality of tools describes functions provided by the plurality of tools.
3 . The method of claim 1 , wherein the call includes a subquery generated by the LLM to aid the LLM in responding to the query.
4 . The method of claim 1 , further comprising:
determining to use a set of tools of the plurality of tools in response to a description of the set of tools matching a portion of the query;
providing a call to each tool of the set of tools with the query;
receiving information from each tool of the set of tools in response to the call to each tool of the set of tools; and
using, by the LLM, the information from each tool to provide a response to the query.
5 . The method of claim 1 , wherein the plurality of tools are included in a plugin that is remote from a copilot hosting the LLM and the copilot accesses the plugin via a network.
6 . The method of claim 1 , wherein the plurality of tools are included in a plurality of plugins remote from a copilot hosting the LLM and the copilot accesses the plurality of plugins via a network.
7 . The method of claim 6 , wherein the plurality of plugins use a standard format and are created by a user for use with the copilot.
8 . The method of claim 1 , wherein the LLM uses a custom dataset for a context of a user that provided the query in generating a tailored response to the context of the user.
9 . The method of claim 1 , wherein the tool of the plurality of tools is a specialized model trained using a LLM pipeline to answer specific questions by retrieving the information from a custom dataset by searching an index of embeddings of the custom dataset and provides the information to the LLM to use in responding to the query.
10 . The method of claim 9 , wherein the LLM pipeline includes:
creating a dataset to train the specialized model by generating questions and answers from a custom dataset;
using the questions and answers to train the specialized model; and
using metrics to evaluate a quality of the questions and answers and a quality of the specialized model.
11 . A device comprising:
a memory to store data and instructions; and
a processor operable to communicate with the memory, wherein the processor is operable to:
receive, at a large language model (LLM), a query and a description of a plurality of tools available for use with the LLM;
determine, by the large language model, to use a tool of the plurality of tools in response to a description of the tool matching a portion of the query;
provide a call to the tool with the query;
receive information from the tool in response to the call;
use, by the LLM, the information in providing a response to the query; and
output the response to the query.
12 . The device of claim 11 , wherein each tool of the plurality of tools performs a function that provides the LLM additional capabilities in providing a response to the query and the description of the plurality of tools describes functions provided by the plurality of tools.
13 . The device of claim 11 , wherein the call includes a subquery generated by the LLM to aid the LLM in responding to the query.
14 . The device of claim 11 , wherein the processor is further operable to:
determine to use a set of tools of the plurality of tools in response to a description of the set of tools matching a portion of the query;
provide a call to each tool of the set of tools with the query;
receive information from each tool of the set of tools in response to the call to each tool of the set of tools; and
use, by the LLM, the information from each tool to provide a response to the query.
15 . The device of claim 11 , wherein the plurality of tools are included in a plugin that is remote from a copilot hosting the LLM and the copilot accesses the plugin via a network.
16 . The device of claim 11 , wherein the plurality of tools are included in a plurality of plugins remote from a copilot hosting the LLM and the copilot accesses the plurality of plugins via a network.
17 . The device of claim 16 , wherein the plurality of plugins use a standard format and are created by a user for use with the copilot.
18 . The device of claim 11 , wherein the LLM uses a custom dataset for a context of a user that provided the query in generating a tailored response to the context of the user.
19 . The device of claim 11 , wherein the tool of the plurality of tools is a specialized model trained using a LLM pipeline to answer specific questions by retrieving the information from a custom dataset by searching an index of embeddings of the custom dataset and provides the information to the LLM to use in responding to the query.
20 . The device of claim 19 , wherein the LLM pipeline includes:
creating a dataset to train the specialized model by generating questions and answers from a custom dataset;
using the questions and answers to train the specialized model; and
using metrics to evaluate a quality of the questions and answers and a quality of the specialized model.