Automatic query generation for large language models
In some embodiments, a method determines a first representation for an entity that received a request. Second representations are searched in a prompt store to retrieve a second representation that is determined to match the first representation. A prompt template for a model is associated with the second representation. The method searches for relevant documents for the request in a knowledge base store and retrieves information from a document that is considered relevant to the request. The information provides context for the request. The method inserts at least a portion of the information into the prompt template to generate a prompt that is based on the context and submits the prompt to the model to receive a response. The method responds to the request using the response.
1. A method comprising:
determining a first representation for an entity that received a request;
searching second representations in a prompt store to retrieve a second representation that is determined to match the first representation, wherein a prompt template for a model is associated with the second representation, and wherein the prompt template includes information for the entity that is used to cause the response to be in an entity voice for the entity that defines characteristics for a personality that is associated with the entity;
searching for relevant documents for the request in a knowledge base store;
retrieving information from a document that is considered relevant to the request, wherein the information provides context for the request;
inserting at least a portion of the information into the prompt template to generate a prompt that is based on the context;
submitting the prompt to the model to receive a response, wherein the response is in the entity voice based on providing the information for the entity voice to the model in the prompt; and
responding to the request using the response.
2. The method of claim 1 , wherein searching second representations comprises:
searching for second representations that are similar to the first representation in a lower dimensional space compared to the request.
3. The method of claim 1 , wherein:
the first representation and the second representations are embeddings in a space, and
a distance of the second representations to the first representation is used to select the second representation.
4. The method of claim 1 , wherein:
the prompt template includes one or more fields where the information can be inserted.
5. The method of claim 1 , wherein searching for relevant documents for the request comprises:
determining a conversation between a user and an agent for the entity, and
using text from the conversation to search for the relevant documents.
6. The method of claim 5 , wherein the agent is an automated agent that automatically generates responses in the conversation.
7. The method of claim 5 , wherein searching for relevant documents for the request comprises:
generating an embedding from the conversation; and
searching embeddings for the relevant documents using the embedding from the conversation to select the document.
8. The method of claim 7 , wherein a distance of the embedding from the conversation to an embedding for the document is used to select the document.
9. The method of claim 1 , wherein retrieving information from the document comprises:
retrieving information for a field in the prompt from the document based on a type of information that is associated with the field.
10. The method of claim 1 , further comprising:
generating a score for the response;
comparing the score to a threshold; and
automatically sending the response to a consumer device that sent the request when the score meets the threshold.
11. The method of claim 10 , further comprising:
when the score does not meet the threshold, verifying whether the response should be sent and editing the response before sending the response.
12. The method of claim 1 , further comprising:
performing training to generate prompt templates with embeddings that are stored in the prompt store.
13. The method of claim 12 , wherein performing training comprises:
generating a current embedding from information for the entity;
searching the prompt store for an embedding that is determined to match with current embedding; and
when an embedding is determined from the prompt store to match the current embedding, storing the current embedding with a prompt template that is associated with the embedding.
14. The method of claim 13 , wherein performing training comprises:
when an embedding is not determined from the prompt store to match the current embedding, searching the prompt store for an embedding that is determined to be similar with current embedding but not the same;
rating the embedding to determine if the embedding is similar; and
storing the current embedding with a prompt template that is associated with the embedding when the embedding is determined to be similar.
15. The method of claim 13 , wherein performing training comprises:
editing a prompt template for the embedding based on a difference between the embedding and the current embedding to generate a new prompt template, wherein the prompt template that is stored with the current embedding is the new prompt template.
16. The method of claim 1 , wherein the characteristics for the personality that are used differ for different entities with different personalities.
17. A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for:
determining a first representation for an entity that received a request;
searching second representations in a prompt store to retrieve a second representation that is determined to match the first representation, wherein a prompt template for a model is associated with the second representation, and wherein the prompt template includes information for the entity that is used to cause the response to be in an entity voice for the entity that defines characteristics for a personality that is associated with the entity;
searching for relevant documents for the request in a knowledge base store;
retrieving information from a document that is considered relevant to the request, wherein the information provides context for the request;
inserting at least a portion of the information into the prompt template to generate a prompt that is based on the context;
submitting the prompt to the model to receive a response, wherein the response is in the entity voice based on providing the information for the entity voice to the model in the prompt; and
responding to the request using the response.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the prompt template includes information for the entity that is used to cause the response to be in a voice for the entity.
19. The non-transitory computer-readable storage medium of claim 17 , wherein searching for relevant documents for the request comprises:
determining a conversation between a user and an agent for the entity, and
using text from the conversation to search for the relevant documents.
20. An apparatus comprising:
one or more computer processors; and
a computer-readable storage medium comprising instructions for controlling the one or more computer processors to be operable for:
determining a first representation for an entity that received a request;
searching second representations in a prompt store to retrieve a second representation that is determined to match the first representation, wherein a prompt template for a model is associated with the second representation, and wherein the prompt template includes information for the entity that is used to cause the response to be in an entity voice for the entity that defines characteristics for a personality that is associated with the entity;
searching for relevant documents for the request in a knowledge base store;
retrieving information from a document that is considered relevant to the request, wherein the information provides context for the request;
inserting at least a portion of the information into the prompt template to generate a prompt that is based on the context;
submitting the prompt to the model to receive a response, wherein the response is in the entity voice based on providing the information for the entity voice to the model in the prompt; and
responding to the request using the response.