SYSTEMS AND METHODS FOR QUERY-FOCUSED SUMMARIZATION
Embodiments described herein provide a query-focused summarization model that employs a single or dual encoder model. A two-step approach may be adopted that first extracts parts of the source document and then synthesizes the extracted segments into a final summary. In another embodiment, an end-to-end approach may be adopted that splits the source document into overlapping segments, and then concatenates encodings into a single embedding sequence for the decoder to output a summary.
1 . A system for user-query guided summarization of documents, the system comprising:
a communication interface that receives a document and a user query posing a question relating to a content of the document;
a memory storing a plurality of processor-executable instructions; and
a processor executing the instructions to perform operations comprising:
splitting the document into a plurality of overlapping segments;
encoding, via a text encoder, each overlapping segment separately appended with the user query into a respective encoding;
concatenating encodings of the plurality of overlapping segments into an embedding sequence; and
generating, via a text decoder, an output summary from the embedding sequence.
2 . The system of claim 1 , wherein an operation of encoding comprises:
applying sparse attention within each overlapping segment without cross-attention between encoded segments.
3 . The system of claim 1 , wherein an operation of generating the output summary comprises:
attending, by the decoder, to all encoded segments jointly.
4 . The system of claim 1 , wherein the encoder and the decoder are built on a Transformer model.
5 . The system of claim 1 , wherein the source document contains query-relevant sections that are related to the input query, and
wherein the query-relevant sections have a size within a processing capacity of a Transformer model.
6 . The system of claim 1 , wherein the text encoder is a single-encoder model that jointly encodes the overlapping segment appended with the user query.
7 . The system of claim 1 , wherein the text encoder is a double-encoder model that separately encodes the user query and the overlapping segment into a query embedding and a passage embedding, respectively.
8 . A method for user-query guided summarization of documents, the system comprising:
receiving, via a communication interface, a document and a user query posing a question relating to a content of the document;
splitting the document into a plurality of overlapping segments;
encoding, via a text encoder, each overlapping segment separately appended with the user query into a respective encoding;
concatenating encodings of the plurality of overlapping segments into an embedding sequence; and
generating, via a text decoder, an output summary from the embedding sequence.
9 . The method of claim 8 , wherein an operation of encoding comprises:
applying sparse attention within each overlapping segment without cross-attention between encoded segments.
10 . The method of claim 8 , wherein an operation of generating the output summary comprises:
attending, by the decoder, to all encoded segments jointly.
11 . The method of claim 8 , wherein the encoder and the decoder are built on a Transformer model.
12 . The method of claim 8 , wherein the source document contains query-relevant sections that are related to the input query, and
wherein the query-relevant sections have a size within a processing capacity of a Transformer model.
13 . The method of claim 8 , wherein the text encoder is a single-encoder model that jointly encodes the overlapping segment appended with the user query.
14 . The method of claim 8 , wherein the text encoder is a double-encoder model that separately encodes the user query and the overlapping segment into a query embedding and a passage embedding, respectively.
15 . A non-transitory processor-readable storage medium storing a plurality of processor-executable instructions for user-query guided summarization of documents, the instructions being executed by one or more hardware processors to perform operations comprising:
receiving, via a communication interface, a document and a user query posing a question relating to a content of the document;
splitting the document into a plurality of overlapping segments;
encoding, via a text encoder, each overlapping segment separately appended with the user query into a respective encoding;
concatenating encodings of the plurality of overlapping segments into an embedding sequence; and
generating, via a text decoder, an output summary from the embedding sequence.
16 . The non-transitory processor-readable storage medium of claim 15 , wherein an operation of encoding comprises:
applying sparse attention within each overlapping segment without cross-attention between encoded segments.
17 . The non-transitory processor-readable storage medium of claim 15 , wherein an operation of generating the output summary comprises:
attending, by the decoder, to all encoded segments jointly.
18 . The non-transitory processor-readable storage medium of claim 15 , wherein the encoder and the decoder are built on a Transformer model.
19 . The non-transitory processor-readable storage medium of claim 15 , wherein the source document contains query-relevant sections that are related to the input query, and
wherein the query-relevant sections have a size within a processing capacity of a Transformer model.
20 . The non-transitory processor-readable storage medium of claim 15 , wherein the text encoder is a single-encoder model that jointly encodes the overlapping segment appended with the user query.