Domain-specific question answering with context reduction for decision making
Methods and systems for context reduction include identifying a context document relating to a query. A number of sentences of the context document to preserve is determined. The sentences of the context document are ranked according to respective similarities between the sentences and the query. A reduced context is generated that preserves the determined number of highest ranked sentences of the context document and eliminates other sentences from the context document. The query is executed with a language model, including the reduced context in a prompt, to generate a response.
1 . A computer-implemented method for context reduction, comprising:
identifying a context document relating to a query;
training a policy using reinforcement learning to balance a token ratio and an accuracy difference, including a reward function:
R
=
-
(
1
-
α
)
τ
+
α
(
2
r
-
r
*
)
where τ=t/T is a token ratio between a number of tokens t in the reduced context and a number of tokens T in the context document, r is a score representing accuracy of an output generated by the language model using the reduced context, r* is a score representing accuracy of an output generated by the language model using the context document, and α is a weighting parameter;
determining a number of sentences of the context document to preserve, including applying the query and the context document to the policy to select a proportion of the context document to preserve;
ranking the sentences of the context document according to respective similarities between the sentences and the query;
generating a reduced context that preserves the determined number of highest ranked sentences of the context document and eliminates other sentences from the context document; and
executing the query with a language model, including the reduced context in a prompt, to generate a response.
2 . The method of claim 1 , wherein generating the reduced context includes eliminating sentences before a first of the highest ranked sentences and sentences after a last of the highest ranked sentences in the context document.
3 . The method of claim 1 , wherein generating the reduced context further includes performing text reduction on sentences other than the highest ranked sentences that occur after a first of the highest ranked sentences and before a last of the highest ranked sentences in the context document.
4 . The method of claim 1 , wherein generating the reduced context includes preserving an order of the highest-ranked sentences from the context document.
5 . The method of claim 1 , wherein ranking the sentences includes determining a similarity score between the query and each respective sentence.
6 . The method of claim 5 , wherein determining the similarity score includes determining a cosine similarity between embeddings of the query and each respective sentence.
7 . The method of claim 1 , wherein the context document includes subject matter that was not used in training the language model.
8 . A system for context reduction, comprising:
a hardware processor; and
a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to:
identify a context document relating to a query;
train a policy using reinforcement learning to balance a token ratio and an accuracy difference, including a reward function:
R
=
-
(
1
-
α
)
τ
+
α
(
2
r
-
r
*
)
where τ=t/T is a token ratio between a number of tokens t in the reduced context and a number of tokens T in the context document, r is a score representing accuracy of an output generated by the language model using the reduced context, r* is a score representing accuracy of an output generated by the language model using the context document, and α is a weighting parameter;
determine a number of sentences of the context document to preserve, including application of the query and the context document to the policy that selects a proportion of the context document to preserve;
rank the sentences of the context document according to respective similarities between the sentences and the query;
generate a reduced context that preserves the determined number of highest ranked sentences of the context document and eliminates other sentences from the context document; and
execute the query with a language model, including the reduced context in a prompt, to generate a response.
9 . The system of claim 8 , wherein the computer program further causes the hardware processor to eliminate sentences before a first of the highest ranked sentences and sentences after a last of the highest ranked sentences in the context document.
10 . The system of claim 8 , wherein the computer program further causes the hardware processor to perform text reduction on sentences other than the highest ranked sentences that occur after a first of the highest ranked sentences and before a last of the highest ranked sentences in the context document.
11 . The system of claim 8 , wherein the computer program further causes the hardware processor to preserve an order of the highest-ranked sentences from the context document in the reduced context.
12 . The system of claim 8 , wherein the computer program further causes the hardware processor to determine a similarity score between the query and each respective sentence.
13 . The system of claim 12 , wherein the computer program further causes the hardware processor to determine a cosine similarity between embeddings of the query and each respective sentence as the similarity score.
14 . The system of claim 8 , wherein the context document includes subject matter that was not used in training the language model.