Natural language generation using knowledge graph incorporating textual summaries
Some techniques relate to generating a knowledge graph including textual passages and textual summaries usable for producing an answer to a question relating to a domain. A corpus of textual information is received. Textual passages and descriptions of associations between textual passages from a first language model are obtained by providing at least a portion of the corpus of textual information to the first language model. Textual summaries corresponding to textual passages are obtained from a second language model. A knowledge graph is generated based on the textual passages, descriptions of associations between the textual passages, and textual summaries. The knowledge graph is stored in a non-transitory computer readable medium.
1 . A computer-implemented method for generating a knowledge graph including textual passages and textual summaries usable for producing an answer to a question relating to a domain, the method comprising:
receiving a corpus of textual information;
obtaining textual passages and descriptions of associations between textual passages from a first language model by providing at least a portion of the corpus of textual information to the first language model, wherein the first language model is pre-trained to produce an output of textual passages and descriptions of associations between textual passages based on an input to the first language model without reference to a pre-determined ontology;
obtaining textual summaries corresponding to textual passages from a second language model;
generating a knowledge graph based on the textual passages, descriptions of associations between the textual passages, and textual summaries;
storing the knowledge graph in a non-transitory computer readable medium;
receiving a natural language textual sequence representing a textual input;
retrieving a set of textual passages from the knowledge graph by comparing the natural language textual sequence against the textual passages in the knowledge graph using a cosine similarity metric, wherein the set of textual passages are retrieved by ranking the textual passages based on the comparison, and selecting the set of textual passages having a value over a threshold value; and
obtaining a textual output to the textual input from the second language model by providing the second language model with the natural language textual sequence, the set of textual passages, and the textual summaries.
2 . The method of claim 1 , wherein the generated knowledge graph includes a tree corresponding to a first document of the textual information, a root of the tree corresponding to a summary of the first document, nodes of the tree corresponding to textual summaries of portions of the first document, and leaves of the tree corresponding to textual passages of the first document.
3 . The method of claim 1 , further comprising:
pre-processing the portion of the corpus of textual information using a context-aware splitting model to produce output of a size less than a context window of the first language model.
4 . The method of claim 1 , wherein the first language model is a decoder-only model.
5 . The method of claim 1 , wherein the output of textual passages and associations between textual passages is formatted in a JavaScript Object Notation format and the knowledge graph is stored in a relational database.
6 . A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing instructions, which, when executed by the system, cause the system to perform a set of actions including:
receiving a corpus of textual information;
obtaining textual passages and descriptions of associations between textual passages from a first language model by providing at least a portion of the corpus of textual information to the first language model, wherein the first language model is pre-trained to produce an output of textual passages and descriptions of associations between textual passages based on an input to the first language model without reference to a pre-determined ontology;
obtaining textual summaries corresponding to textual passages from a second language model;
generating a knowledge graph based on the textual passages, descriptions of associations between the textual passages, and textual summaries;
storing the knowledge graph in a non-transitory computer readable medium;
receiving a natural language textual sequence representing a textual input;
retrieving a set of textual passages from the knowledge graph by comparing the natural language textual sequence against the textual passages in the knowledge graph using a cosine similarity metric, wherein the set of textual passages are retrieved by ranking the textual passages based on the comparison, and selecting the set of textual passages having a value over a threshold value; and
obtaining a textual output to the textual input from the second language model by providing the second language model with the natural language textual sequence, the set of textual passages, and the textual summaries.
7 . The system of claim 6 , wherein the generated knowledge graph includes a tree corresponding to a first document of the textual information, a root of the tree corresponding to a summary of the first document, nodes of the tree corresponding to textual summaries of portions of the first document, and leaves of the tree corresponding to textual passages of the first document.
8 . The system of claim 6 , wherein the set of actions further includes:
pre-processing the portion of the corpus of textual information using a context-aware splitting model to produce output of a size less than a context window of the first language model.
9 . The system of claim 6 , wherein the first language model is a decoder-only model.
10 . The system of claim 6 , wherein the output of textual passages and associations between textual passages is formatted in a JavaScript Object Notation format and the knowledge graph is stored in a relational database.
11 . A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform part or all of one or more methods disclosed herein:
receiving a corpus of textual information;
obtaining textual passages and descriptions of associations between textual passages from a first language model by providing at least a portion of the corpus of textual information to the first language model, wherein the first language model is pre-trained to produce an output of textual passages and descriptions of associations between textual passages based on an input to the first language model without reference to a pre-determined ontology;
obtaining textual summaries corresponding to textual passages from a second language model;
generating a knowledge graph based on the textual passages, descriptions of associations between the textual passages, and textual summaries;
storing the knowledge graph in a non-transitory computer readable medium;
receiving a natural language textual sequence representing a textual input;
retrieving a set of textual passages from the knowledge graph by comparing the natural language textual sequence against the textual passages in the knowledge graph using a cosine similarity metric, wherein the set of textual passages are retrieved by ranking the textual passages based on the comparison, and selecting the set of textual passages having a value over a threshold value; and
obtaining a textual output to the textual input from the second language model by providing the second language model with the natural language textual sequence, the set of textual passages, and the textual summaries.
12 . The computer-program product of claim 11 , wherein the generated knowledge graph includes a tree corresponding to a first document of the textual information, a root of the tree corresponding to a summary of the first document, nodes of the tree corresponding to textual summaries of portions of the first document, and leaves of the tree corresponding to textual passages of the first document.
13 . The computer-program product of claim 11 , wherein set of actions further includes:
pre-processing the portion of the corpus of textual information using a context-aware splitting model to produce output of a size less than a context window of the first language model.
14 . The computer-program product of claim 11 , wherein the first language model is a decoder-only model.
15 . The computer-program product of claim 11 , wherein the output of textual passages and associations between textual passages is formatted in a JavaScript Object Notation format and the knowledge graph is stored in a relational database.