Natural language generation using knowledge graph incorporating textual summaries
Some embodiments relate to receiving a natural-language textual sequence representing; retrieving, from a knowledge graph, a first textual passage and a second textual passage based on rankings with respect to the natural-language textual sequence, a first textual summary summarizing textual information in a first vicinity of the first textual passage, and a second textual summary summarizing textual information in a vicinity of the second textual passage; obtaining the textual output in response to the textual input using a language model by encoding a first intermediate output based on the natural-language textual sequence, the first textual passage, and the first textual summary, encoding a second intermediate output based on the natural language textual sequence, the second textual passage, and the second textual summary, and decoding a concatenation of the first intermediate output and the second intermediate output; and providing an output to a user based on the textual output.
1 . A computer-implemented method for producing a textual output in response to a textual input relating to a domain, the method comprising:
receiving a natural-language textual sequence representing the textual input;
retrieving, from a knowledge graph associated with the domain, a first textual passage and a second textual passage based on rankings with respect to the natural-language textual sequence, a first textual summary summarizing textual information in a first vicinity of the first textual passage, and a second textual summary summarizing textual information in a second vicinity of the second textual passage, wherein:
the first vicinity and the second vicinity are determined based on a number of tokens that are before and/or after the first textual passage and the second textual passage, respectively, and the number of tokens is determined based on a length of a document or a determination of an amount of text that has a relationship with textual passages, wherein tokens represent words, subwords, or characters of the textual passages,
the tokens are converted into high dimensional input sequence of vectors where a high dimensional input sequence of vector numerically represents a token and relationships between the plurality of tokens in a representation that is processed by a language model,
the language model includes one or more encoders and decoders to process the high dimensional input sequence of vectors to generate an output sequence used to determine an output of the language model, wherein each encoder includes plurality of encoder layers and each decoder includes plurality of decoder layers, and
the knowledge graph is generated based on textual passages, descriptions of associations between the textual passages, and textual summaries obtained by providing at least a portion of a corpus of textual information to the language model, the corpus of textual information includes a plurality of documents, wherein the textual passages include text in the document, the corpus of textual information, or variations on the text in the document or the corpus of textual information;
obtaining the textual output in response to the textual input using the language model by encoding a first intermediate outputs based on the natural-language textual sequence, the first textual passage, and the first textual summary, encoding a second intermediate outputs based on the natural-language textual sequence, the second textual passage, and the second textual summary, and ranking the first intermediate outputs and the second intermediate outputs and providing the language model with a subset of the ranked first intermediate outputs and second intermediate outputs for decoding a concatenation of the subset of the first intermediate outputs and the second intermediate outputs; and
providing the output to a user on a user interface based on the textual output.
2 . The method of claim 1 , wherein retrieving further includes retrieving, from the knowledge graph associated with the domain, a third textual passage based on a ranking with respect to its relationship to the natural-language textual sequence and a third textual summary summarizing textual information in a vicinity of the third textual passage.
3 . The method of claim 1 , wherein retrieving further includes retrieving, from the knowledge graph associated with the domain, a fourth textual summary based on a second ranking with respect to the natural-language textual sequence, wherein obtaining the textual output using the language model further includes encoding a fourth intermediate output based on the natural-language textual sequence and the fourth textual summary.
4 . The method of claim 3 , wherein the concatenation of intermediate outputs includes at least one of a third intermediate output and the fourth intermediate output.
5 . The method of claim 1 , wherein the knowledge graph includes a hierarchical structure corresponding to the document, with a root representing a summary of the document, nodes representing textual summaries of sections, and leaves representing textual passages, wherein retrieving further includes retrieving the summary of the document corresponding to the root and a fifth textual summary summarizing information in a vicinity of the first textual passage.
6 . The method of claim 5 , wherein encoding the first intermediate output is further based on the summary of the document and the fifth textual summary, wherein the first vicinity includes a first set of textual information and the second vicinity includes the first set of textual information and an additional second set of textual information.
7 . The method of claim 1 , wherein the language model is trained to process multiple textual passages and summaries jointly to generate a more contextually relevant textual output.
8 . A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform a set of actions for producing a textual output in response to a textual input relating to a domain, the set of actions comprising:
receiving a natural-language textual sequence representing the textual input;
retrieving, from a knowledge graph associated with the domain, a first textual passage and a second textual passage based on rankings with respect to the natural-language textual sequence, a first textual summary summarizing textual information in a first vicinity of the first textual passage, and a second textual summary summarizing textual information in a second vicinity of the second textual passage, wherein:
the first vicinity and the second vicinity are determined based on a number of tokens that are before and/or after the first textual passage and the second textual passage, respectively, and the number of tokens is determined based on a length of a document or a determination of an amount of text that has a relationship with textual passages, wherein tokens represent words, subwords, or characters of the textual passages,
the tokens are converted into high dimensional input sequence of vectors where a high dimensional input sequence of vector numerically represents a token and relationships between the plurality of tokens in a representation that is processed by a language model,
the language model includes one or more encoders and decoders to process the high dimensional input sequence of vectors to generate an output sequence used to determine an output of the language model, wherein each encoder includes plurality of encoder layers and each decoder includes plurality of decoder layers, and
the knowledge graph is generated based on textual passages, descriptions of associations between the textual passages, and textual summaries obtained by providing at least a portion of a corpus of textual information to the language model, the corpus of textual information includes a plurality of documents, wherein the textual passages include text in the document, the corpus of textual information, or variations on the text in the document or the corpus of textual information;
obtaining the textual output in response to the textual input using the language model by encoding a first intermediate outputs based on the natural-language textual sequence, the first textual passage, and the first textual summary, encoding a second intermediate outputs based on the natural-language textual sequence, the second textual passage, and the second textual summary, and re-ranking the first intermediate outputs and the second intermediate outputs and providing the language model with a subset of the ranked first intermediate outputs and second intermediate outputs for decoding a concatenation of the subset of the first intermediate outputs and the second intermediate outputs; and
providing the output to a user on a user interface based on the textual output.
9 . The computer-program product of claim 8 , wherein retrieving further includes retrieving, from the knowledge graph associated with the domain, a third textual passage based on a ranking with respect to its relationship to the natural-language textual sequence and a third textual summary summarizing textual information in a vicinity of the third textual passage.
10 . The computer-program product of claim 8 , wherein retrieving further includes retrieving, from the knowledge graph associated with the domain, a fourth textual summary based on a second ranking with respect to the natural-language textual sequence, wherein obtaining the textual output using the language model further includes encoding a fourth intermediate output based on the natural-language textual sequence and the fourth textual summary.
11 . The computer-program product of claim 10 , wherein the concatenation of intermediate outputs includes at least one of a third intermediate output and the fourth intermediate output.
12 . The computer-program product of claim 8 , wherein the knowledge graph includes a hierarchical structure corresponding to the document, with a root representing a summary of the document, nodes representing textual summaries of sections, and leaves representing textual passages, wherein retrieving further includes retrieving the summary of the document corresponding to the root and a fifth textual summary summarizing information in a vicinity of the first textual passage.
13 . The computer-program product of claim 12 , wherein encoding the first intermediate output is further based on the summary of the document and the fifth textual summary, wherein the first vicinity includes a first set of textual information and the second vicinity includes the first set of textual information and an additional second set of textual information.
14 . The computer-program product of claim 8 , wherein the language model is trained to process multiple textual passages and summaries jointly to generate a more contextually relevant textual output.
15 . A system comprising:
one or more processors;
one or more non-transitory computer-readable media storing instructions, which, when executed by the system, cause the system to perform a set of actions for producing a textual output in response to a textual input relating to a domain, the set of actions comprising:
receiving a natural-language textual sequence representing the textual input;
retrieving, from a knowledge graph associated with the domain, a first textual passage and a second textual passage based on rankings with respect to the natural-language textual sequence, a first textual summary summarizing textual information in a first vicinity of the first textual passage, and a second textual summary summarizing textual information in a second vicinity of the second textual passage, wherein:
the first vicinity and the second vicinity are determined based on a number of tokens that are before and/or after the first textual passage and the second textual passage, respectively, and the number of tokens is determined based on a length of a document or a determination of an amount of text that has a relationship with textual passages, wherein tokens represent words, subwords, or characters of the textual passages,
the tokens are converted into high dimensional input sequence of vectors where a high dimensional input sequence of vector numerically represents a token and relationships between the plurality of tokens in a representation that is processed by a language model,
the language model includes one or more encoders and decoders to process the high dimensional input sequence of vectors to generate an output sequence used to determine an output of the language model, wherein each encoder includes plurality of encoder layers and each decoder includes plurality of decoder layers, and
the knowledge graph is generated based on textual passages, descriptions of associations between the textual passages, and textual summaries obtained by providing at least a portion of a corpus of textual information to the language model, the corpus of textual information includes a plurality of documents, wherein the textual passages include text in the document, the corpus of textual information, or variations on the text in the document or the corpus of textual information;
obtaining the textual output in response to the textual input using the language model by encoding a first intermediate outputs based on the natural-language textual sequence, the first textual passage, and the first textual summary, encoding a second intermediate outputs based on the natural-language textual sequence, the second textual passage, and the second textual summary, and ranking the first intermediate outputs and the second intermediate outputs and providing the language model with a subset of the ranked first intermediate outputs and second intermediate outputs for decoding a concatenation of the subset of the first intermediate outputs and the second intermediate outputs; and
providing the output to a user on a user interface based on the textual output.
16 . The system of claim 15 , wherein retrieving further includes retrieving, from the knowledge graph associated with the domain, a third textual passage based on a ranking with respect to its relationship to the natural-language textual sequence and a third textual summary summarizing textual information in a vicinity of the third textual passage.
17 . The system of claim 15 , wherein retrieving further includes retrieving, from the knowledge graph associated with the domain, a fourth textual summary based on a second ranking with respect to the natural-language textual sequence, wherein obtaining the textual output using the language model further includes encoding a fourth intermediate output based on the natural-language textual sequence and the fourth textual summary.
18 . The system of claim 17 , wherein the concatenation of intermediate outputs includes at least one of a third intermediate output and the fourth intermediate output.
19 . The system of claim 15 , wherein the knowledge graph includes a hierarchical structure corresponding to the document, with a root representing a summary of the document, nodes representing textual summaries of sections, and leaves representing textual passages, wherein retrieving further includes retrieving the summary of the document corresponding to the root and a fifth textual summary summarizing information in a vicinity of the first textual passage.
20 . The system of claim 19 , wherein encoding the first intermediate output is further based on the summary of the document and the fifth textual summary, wherein the first vicinity includes a first set of textual information and the second vicinity includes the first set of textual information and an additional second set of textual information.