NATURAL LANGUAGE GENERATION USING KNOWLEDGE GRAPH INCORPORATING TEXTUAL SUMMARIES
Techniques relate to receiving a natural-language textual sequence representing the textual input; retrieving, from a knowledge graph associated with the domain: a first textual passage based on a first ranking regarding the natural-language textual sequence, a first textual summary associated with the first textual passage, a second textual passage based on a first ranking with respect to a comparison of the second textual passage and the natural-language textual sequence a second textual summary associated with the second textual passage, and a third textual summary based on a second ranking regarding the natural-language textual sequence; obtaining the textual output in response to the textual input from a language model by providing input to the language model using the natural-language textual sequence, the first textual passage, the first textual summary, the second textual passage, the second textual summary, and the third textual summary; and providing an output based on the textual output.
1 . A computer-implemented method for producing a textual output in response to a textual input relating to a domain, the method comprising:
receiving a natural-language textual sequence representing the textual input from a user;
retrieving, from a knowledge graph associated with the domain:
a first textual passage based on a first ranking with respect to the natural-language textual sequence,
a first textual summary summarizing textual information in a first vicinity of the first textual passage,
a second textual passage based on a first ranking with respect to a comparison of the second textual passage and the natural-language textual sequence,
a second textual summary summarizing textual information in a second vicinity of the second textual passage, and
a third textual summary based on a second ranking with respect to the natural-language textual sequence, wherein:
the knowledge graph is generated based on textual passages, descriptions of associations between the textual passages, and textual summaries obtained by providing at least a portion of a corpus of textual information to a language model, the corpus of textual information includes a plurality of documents, and
the language model is provided with previously generated textual summaries at a higher level or a lower level to reduce duplication of information included in the textual summaries between a plurality of levels, wherein the higher level corresponds to the second textual summary, and the lower level corresponds to the first textual summary, where the second vicinity includes more text than the first vicinity and includes the text of the first vicinity;
obtaining the textual output in response to the textual input from the language model by providing input to the language model based on the natural-language textual sequence, the first textual passage, the first textual summary, the second textual passage, the second textual summary, and the third textual summary; and
providing an output based on the textual output.
2 . The method of claim 1 , wherein the language model is an encoder-decoder model that encodes at least the first textual passage, the second textual passage, and the third textual summary separately into respective intermediate representations.
3 . The method of claim 2 , wherein the language model re-ranks the intermediate representations before concatenating a subset of them for decoding to generate the textual output.
4 . The method of claim 3 , wherein the subset of intermediate representations includes an encoding based on the natural-language textual sequence, the first textual passage, and the first textual summary.
5 . The method of claim 1 , wherein the first textual passage is from a first document, and the first textual summary is a summary of a portion of the first document, further retrieving, from the knowledge graph, a first document summary that summarizes the first document and incorporating it into the input to the language model.
6 . The method of claim 1 , wherein the knowledge graph includes a tree structure where a root corresponds to a summary of a document, nodes correspond to textual summaries of sections of the document, and leaves correspond to textual passages of the document, and wherein the first textual summary corresponds to a node immediately above a leaf corresponding to the first textual passage.
7 . The method of claim 1 , wherein the knowledge graph is stored in a relational database using a JavaScript Object Notation (JSON) format without relying on a predefined ontology.
8 . A non-transitory computer readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a set of actions for producing a textual output in response to a textual input relating to a domain, the set of actions including:
receiving a natural-language textual sequence representing the textual input from a user;
retrieving, from a knowledge graph associated with the domain:
a first textual passage based on a first ranking with respect to the natural-language textual sequence,
a first textual summary summarizing textual information in a first vicinity of the first textual passage,
a second textual passage based on a first ranking with respect to a comparison of the second textual passage and the natural-language textual sequence,
a second textual summary summarizing textual information in a second vicinity of the second textual passage, and
a third textual summary based on a second ranking with respect to the natural-language textual sequence, wherein:
the knowledge graph is generated based on textual passages, descriptions of associations between the textual passages, and textual summaries obtained by providing at least a portion of a corpus of textual information to a language model, the corpus of textual information includes a plurality of documents, and
the language model is provided with previously generated textual summaries at a higher level or a lower level to reduce duplication of information included in the textual summaries between a plurality of levels, wherein the higher level corresponds to the second textual summary, and the lower level corresponds to the first textual summary, where the second vicinity includes more text than the first vicinity and includes the text of the first vicinity;
obtaining the textual output in response to the textual input from the language model by providing input to the language model based on the natural-language textual sequence, the first textual passage, the first textual summary, the second textual passage, the second textual summary, and the third textual summary; and
providing an output based on the textual output.
9 . The non-transitory computer readable medium of claim 8 , wherein the language model is an encoder-decoder model that encodes at least the first textual passage, the second textual passage, and the third textual summary separately into respective intermediate representations.
10 . The non-transitory computer readable medium of claim 9 , wherein the language model re-ranks the intermediate representations before concatenating a subset of them for decoding to generate the textual output.
11 . The non-transitory computer readable medium of claim 10 , wherein the subset of intermediate representations includes an encoding based on the natural-language textual sequence, the first textual passage, and the first textual summary.
12 . The non-transitory computer readable medium of claim 8 , wherein the first textual passage is from a first document, and the first textual summary is a summary of a portion of the first document, further retrieving, from the knowledge graph, a first document summary that summarizes the first document and incorporating it into the input to the language model.
13 . The non-transitory computer readable medium of claim 8 , wherein the knowledge graph includes a tree structure where a root corresponds to a summary of a document, nodes correspond to textual summaries of sections of the document, and leaves correspond to textual passages of the document, and wherein the first textual summary corresponds to a node immediately above a leaf corresponding to the first textual passage.
14 . The non-transitory computer readable medium of claim 8 , wherein the knowledge graph is stored in a relational database using a JavaScript Object Notation (JSON) format without relying on a predefined ontology.
15 . A system including at least one processor and at least one computer readable medium including instructions that, when executed by the at least one processor, cause the at least one processor to perform a set of actions for producing a textual output in response to a textual input relating to a domain, the set of actions including:
receiving a natural-language textual sequence representing the textual input from a user;
retrieving, from a knowledge graph associated with the domain: a first textual passage based on a first ranking with respect to the natural-language textual sequence,
a first textual summary summarizing textual information in a first vicinity of the first textual passage,
a second textual passage based on a first ranking with respect to a comparison of the second textual passage and the natural-language textual sequence,
a second textual summary summarizing textual information in a second vicinity of the second textual passage, and
a third textual summary based on a second ranking with respect to the natural-language textual sequence, wherein:
the knowledge graph is generated based on textual passages, descriptions of associations between the textual passages, and textual summaries obtained by providing at least a portion of a corpus of textual information to a language model, the corpus of textual information includes a plurality of documents, and
the language model is provided with previously generated textual summaries at a higher level or a lower level to reduce duplication of information included in the textual summaries between a plurality of levels, wherein the higher level corresponds to the second textual summary, and the lower level corresponds to the first textual summary, where the second vicinity includes more text than the first vicinity and includes the text of the first vicinity;
obtaining the textual output in response to the textual input from the language model by providing input to the language model based on the natural-language textual sequence, the first textual passage, the first textual summary, the second textual passage, the second textual summary, and the third textual summary; and
providing an output based on the textual output.
16 . The system of claim 15 , wherein the language model is an encoder-decoder model that encodes at least the first textual passage, the second textual passage, and the third textual summary separately into respective intermediate representations.
17 . The system of claim 16 , wherein the language model re-ranks the intermediate representations before concatenating a subset of them for decoding to generate the textual output.
18 . The system of claim 17 wherein the subset of intermediate representations includes an encoding based on the natural-language textual sequence, the first textual passage, and the first textual summary.
19 . The system of claim 15 , wherein the first textual passage is from a first document, and the first textual summary is a summary of a portion of the first document, further retrieving, from the knowledge graph, a first document summary that summarizes the first document and incorporating it into the input to the language model.
20 . The system of claim 15 , wherein the knowledge graph includes a tree structure where a root corresponds to a summary of a document, nodes correspond to textual summaries of sections of the document, and leaves correspond to textual passages of the document, and wherein the first textual summary corresponds to a node immediately above a leaf corresponding to the first textual passage.