Systems and methods for query-focused summarization
Embodiments described herein provide a query-focused summarization model that employs a single or dual encoder model. A two-step approach may be adopted that first extracts parts of the source document and then synthesizes the extracted segments into a final summary. In another embodiment, an end-to-end approach may be adopted that splits the source document into overlapping segments, and then concatenates encodings into a single embedding sequence for the decoder to output a summary.
1. A method for query-focused summarization, the method comprising:
receiving, via a communication interface, a source document, an input query posing a question on a content of the source document and a reference summary of the source document;
dividing the source document into a plurality of source passages;
encoding, via an encoder model, a source passage from the plurality of source passages and the input query separated by a delimiter;
generating, by a final layer of the encoder model, a predicted relevance between the source passage and the reference summary based on the encoding;
computing a loss function based on the predicted relevance; and
updating the encoder model based on the computed loss function.
2. The method of claim 1 , further comprising:
receiving a testing query and a testing source document;
generating, via the encoder model, predicted relevance scores corresponding to parts of the testing source document;
extracting at least one part of the testing source document relevant to the testing query based on the predicted relevance scores; and
synthesizes, via an abstractor model, the extracted at least one part into a final summary of the testing source document.
3. The method of claim 1 , wherein the loss function is a mean square value of a number of predicted relevances.
4. The method of claim 1 , wherein the encoder model is a single-encoder model that jointly encodes the source passage appended with the input query.
5. The method of claim 1 , wherein the encoder model is a double-encoder model that separately encodes the input query and the source passage into a query embedding and a passage embedding, respectively.
6. The method of claim 5 , wherein the generating the predicted relevance further comprises calculating a cosine similarity between the query embedding and the passage embedding.
7. The method of claim 5 , wherein the passage embedding is pre-computed and stored for a given testing query.
8. A system for query-focused summarization, the system comprising:
a communication interface that receives a source document, an input query posing a question on a content of the source document and a reference summary of the source document;
a memory storing a plurality of processor-executable instructions; and
a processor executing the instructions to perform operations comprising:
dividing the source document into a plurality of source passages;
encoding, via an encoder model, a source passage from the plurality of source passages and the input query separated by a delimiter;
generating, by a final layer of the encoder model, a predicted relevance between the source passage and the reference summary based on the encoding;
computing a loss function based on the predicted relevance; and
updating the encoder model based on the computed loss function.
9. The system of claim 8 , wherein the operations further comprise:
receiving a testing query and a testing source document;
generating, via the encoder model, predicted relevance scores corresponding to parts of the testing source document;
extracting at least one part of the testing source document relevant to the testing query based on the predicted relevance scores; and
synthesizes, via an abstractor model, the extracted at least one part into a final summary of the testing source document.
10. The system of claim 8 , wherein the loss function is a mean square value of a number of predicted relevances.
11. The system of claim 8 , wherein the encoder model is a single-encoder model that jointly encodes the source passage appended with the input query.
12. The system of claim 8 , wherein the encoder model is a double-encoder model that separately encodes the input query and the source passage into a query embedding and a passage embedding, respectively.
13. The system of claim 12 , wherein an operation of generating the predicted relevance further comprises calculating a cosine similarity between the query embedding and the passage embedding.
14. The system of claim 13 , wherein the passage embedding is pre-computed and stored for a given testing query.
15. A non-transitory computer-readable medium storing a plurality of instructions for query-focused summarization, the instructions executed by one or more processors to perform operations comprising:
receiving, via a communication interface, a source document, an input query posing a question on a content of the source document and a reference summary of the source document;
dividing the source document into a plurality of source passages;
encoding, via an encoder model, a source passage from the plurality of source passages and the input query separated by a delimiter;
generating, by a final layer of the encoder model, a predicted relevance between the source passage and the reference summary based on the encoding;
computing a loss function based on the predicted relevance; and
updating the encoder model based on the computed loss function.
16. The non-transitory computer-readable medium of claim 15 , wherein the operations further comprise:
receiving a testing query and a testing source document;
generating, via the encoder model, predicted relevance scores corresponding to parts of the testing source document;
extracting at least one part of the testing source document relevant to the testing query based on the predicted relevance scores; and
synthesizes, via an abstractor model, the extracted at least one part into a final summary of the testing source document.
17. The non-transitory computer-readable medium of claim 15 , wherein the loss function is a mean square value of a number of predicted relevances.
18. The non-transitory computer-readable medium of claim 15 , wherein the encoder model is a single-encoder model that jointly encodes the source passage appended with the input query.
19. The non-transitory computer-readable medium of claim 15 , wherein the encoder model is a double-encoder model that separately encodes the input query and the source passage into a query embedding and a passage embedding, respectively.
20. The non-transitory computer-readable medium of claim 19 , wherein the generating the predicted relevance further comprises calculating a cosine similarity between the query embedding and the passage embedding.