Optimization of retrieval augmented generation using data-driven templates
Systems and methods are disclosed herein for compressing a prompt. In an example system, an importance score listing is obtained that includes a score indicative of an importance of a plurality of dataset keywords. From the importance score listing, a keyword importance score is identified for a plurality of keywords in a current text fragment, such as a text fragment to be compressed. A set of placeholders in an abstract prompt template is populated based on the current text fragment. The current text fragment is compressed based on the importance of the plurality of keywords in the current text fragment to generate a compressed text fragment. In an example, the compressed text fragment is included in the prompt for transmission to a computing entity, such as a large language model of a generative question-answering system.
1 . A system for compressing a prompt, the system comprising:
a processor; and
a memory device that stores program code structured to cause the processor to:
obtain an importance score listing that includes a score indicative of an importance of a plurality of dataset keywords;
identify, from the importance score listing, a keyword importance score for a plurality of keywords in a current text fragment;
populate a set of placeholders in an abstract prompt template based on the current text fragment; and
compress the populated abstract prompt template based on the keyword importance score for the plurality of keywords in the current text fragment to generate a compressed text fragment, the compressed text fragment included in the prompt for transmission to a computing entity.
2 . The system of claim 1 , wherein the computing entity comprises a large language model (LLM), and
wherein the prompt comprises a query for the LLM to generate an answer thereto.
3 . The system of claim 2 , wherein the current text fragment is obtained from a retrieval augmentation generation (RAG) system in response to the query transmitted thereto.
4 . The system of claim 1 , wherein the program code is further structured to cause the processor to:
identify a set of entities and associated dataset keywords from a text fragment training dataset;
obtain the score for each of the plurality of dataset keywords, the dataset keywords extracted from the text fragment training dataset; and
store, in the importance score listing, each dataset keyword, the score for the dataset keyword, and the entity associated with the dataset keyword.
5 . The system of claim 1 , wherein the program code is further structured to cause the processor to:
identify a set of entities from a text fragment training dataset; and
generate the abstract prompt template that includes a plurality of strings, each string comprising one of the entities of the set of entities and a placeholder for an entity value corresponding to the entity.
6 . The system of claim 5 , wherein the program code is structured to cause the processor to populate the set of placeholders in the abstract prompt template by:
for each of the plurality of keywords, identifying a corresponding keyword entity; and
for each keyword entity, populating a placeholder of a string containing the keyword entity in the abstract prompt template with the keyword corresponding to the keyword entity.
7 . The system of claim 1 , wherein the program code is structured to cause the processor to compress the populated abstract prompt template by:
removing information from the populated abstract prompt template that does not satisfy a budgeting criteria.
8 . The system of claim 7 , wherein the budgeting criteria comprises a compression ratio.
9 . The system of claim 1 , wherein the program code is further structured to cause the processor to:
assign a type to a first keyword of the plurality of keywords; and
transform the first keyword into a second keyword in the compressed text fragment based on the type.
10 . A method for compressing a prompt, comprising:
obtaining an importance score listing that includes a score indicative of an importance of a plurality of dataset keywords;
identifying, from the importance score listing, a keyword importance score for a plurality of keywords in a current text fragment;
populating a set of placeholders in an abstract prompt template based on the current text fragment; and
compressing the populated abstract prompt template based on the keyword importance score for the plurality of keywords in the current text fragment to generate a compressed text fragment, the compressed text fragment included in the prompt for transmission to a computing entity.
11 . The method of claim 10 , wherein the computing entity comprises a large language model (LLM), and
wherein the prompt comprises a query for the LLM to generate an answer thereto.
12 . The method of claim 11 , wherein the current text fragment is obtained from a retrieval augmentation generation (RAG) system in response to the query transmitted thereto.
13 . The method of claim 10 , further comprising:
identifying a set of entities and associated dataset keywords from a text fragment training dataset;
obtaining the score for each of the plurality of dataset keywords, the dataset keywords extracted from the text fragment training dataset; and
storing, in the importance score listing, each dataset keyword, the score for the dataset keyword, and the entity associated with the dataset keyword.
14 . The method of claim 10 , further comprising:
identifying a set of entities from a text fragment training dataset; and
generating the abstract prompt template that includes a plurality of strings, each string comprising one of the entities of the set of entities and a placeholder for an entity value corresponding to the entity.
15 . The method of claim 14 , wherein the populating the set of placeholders in the abstract prompt template comprises:
for each of the plurality of keywords, identifying a corresponding keyword entity; and
for each keyword entity, populating a placeholder of a string containing the keyword entity in the abstract prompt template with the keyword corresponding to the keyword entity.
16 . The method of claim 10 , wherein the compressing the populated abstract prompt template comprises:
removing information from the populated abstract prompt template that does not satisfy a budgeting criteria.
17 . A computer-readable storage medium having computer program code recorded thereon that when executed by at least one processor causes the at least one processor to perform a method comprising:
obtaining an importance score listing that includes a score indicative of an importance of a plurality of dataset keywords;
identifying, from the importance score listing, a keyword importance score for a plurality of keywords in a current text fragment;
populating a set of placeholders in an abstract prompt template based on the current text fragment; and
compressing the populated abstract prompt template based on the keyword importance score for the plurality of keywords in the current text fragment to generate a compressed text fragment, the compressed text fragment included in the prompt for transmission to a computing entity.
18 . The computer-readable storage medium of claim 17 , wherein the computing entity comprises a large language model (LLM), and
wherein the prompt comprises a query for the LLM to generate an answer thereto.
19 . The computer-readable storage medium of claim 18 , wherein the current text fragment is obtained from a retrieval augmentation generation (RAG) system in response to the query transmitted thereto.
20 . The computer-readable storage medium of claim 17 , wherein the method further comprises:
identifying a set of entities and associated dataset keywords from a text fragment training dataset;
obtaining the score for each of the plurality of dataset keywords, the dataset keywords extracted from the text fragment training dataset; and
storing, in the importance score listing, each dataset keyword, the score for the dataset keyword, and the entity associated with the dataset keyword.