SYSTEMS AND METHODS FOR OPEN DOMAIN MULTI-HOP QUESTION ANSWERING
Embodiments described herein provide a fusion-in-decoder (FID) based model (referred to as “PATHID”) for open-domain multi-hop question answering. Specifically, PATHID addresses the gap between the general behavior of the FID model on single-hop and multi-hop question answering, and provides more transparency into the reasoning path. In addition to answer generation, PATHID explicitly models the full reasoning path to resolve the answer with a generative sequence-to-sequence model.
1 . A method for multi-hop question answering and reasoning via a natural language processing (NLP) model, the method comprising:
receiving, via a communication interface, a multi-hop question and a collection of passages;
generating a plurality of input blocks, each of which contains a concatenation of the multi-hop question, a respective title of a respective passage, and a respective context representation of the respective passage;
encoding, via an encoder, the plurality of input blocks into a plurality of encoded input representations;
concatenating the plurality of encoded input representations into a global input representation;
generating, via a decoder in response to the global input representation, a decoded sequence containing a title block, a supporting fact block and an answer block; and
generating an answer to the multi-hop question based on the answer block and a reasoning path accompanying the answer based on the title block and the supporting fact block.
2 . The method of claim 1 , wherein the context representation is generated by inserting special fact tokens that signify starts of a sentence before each sentence of the respective passage.
3 . The method of claim 1 , wherein the decoded sequence contains a linearized sequence of alternating title blocks and supporting fact blocks, and
wherein the alternating title blocks and supporting fact blocks are selected from a sequence of passages indicating a reasoning for locating the answer to the multi-hop question from the collection of passages.
4 . The method of claim 1 , wherein the supporting fact block contains a fact starting token followed by a sequence of fact indicators corresponding to special fact tokens in the context representation.
5 . The method of claim 1 , wherein at least one input block from the plurality of input blocks contains a concatenation of the multi-hop question, a first title of a first passage, a first context representation of the first passage, a second title of a second passage, and a second context representation of the second passage.
6 . The method of claim 1 , wherein the decoded sequence is generated autoregressively per token at each step via a self-attention module, a cross-attention module and a feed-forward module.
7 . The method of claim 1 , wherein the decoded sequence is generated by the decoder in a form of a conditional probability distribution of the decoded sequence conditioned on the global input representation.
8 . The method of claim 7 , further comprising:
computing a loss objective based on an entropy of the conditional probability distribution of the decoded sequence conditioned on the global input representation; and
updating parameters of the encoder and the decoder by minimizing the loss objective.
9 . The method of claim 1 , wherein the answer is generated by parsing the decoded sequence based on an answer indicator.
10 . The method of claim 9 , wherein the reasoning path is generated by:
recursively parsing, the decoded sequence after removing the answer block, based on separator tokens indicating a start of the title block or the supporting fact block; and
reconstructing a title and relevant sentences at each hop of the recursive parsing.
11 . A system for multi-hop question answering and reasoning via a natural language processing (NLP) model, the system comprising:
a communication interface receiving a multi-hop question and a collection of passages;
a memory for storing an encoder and a decoder, and a plurality of processor-executable instructions; and
a processor that executes the plurality of processor-executable instructions to perform operations comprising:
generating a plurality of input blocks, each of which contains a concatenation of the multi-hop question, a respective title of a respective passage, and a respective context representation of the respective passage;
encoding, via an encoder, the plurality of input blocks into a plurality of encoded input representations;
concatenating the plurality of encoded input representations into a global input representation;
generating, via a decoder in response to the global input representation, a decoded sequence containing a title block, a supporting fact block and an answer block; and
generating an answer to the multi-hop question based on the answer block and a reasoning path accompanying the answer based on the title block and the supporting fact block.
12 . The system of claim 11 , wherein the context representation is generated by inserting special fact tokens that signify starts of a sentence before each sentence of the respective passage.
13 . The system of claim 11 , wherein the decoded sequence contains a linearized sequence of alternating title blocks and supporting fact blocks, and
wherein the alternating title blocks and supporting fact blocks are selected from a sequence of passages indicating a reasoning for locating the answer to the multi-hop question from the collection of passages.
14 . The system of claim 11 , wherein the supporting fact block contains a fact starting token followed by a sequence of fact indicators corresponding to special fact tokens in the context representation.
15 . The system of claim 11 , wherein at least one input block from the plurality of input blocks contains a concatenation of the multi-hop question, a first title of a first passage, a first context representation of the first passage, a second title of a second passage, and a second context representation of the second passage.
16 . The system of claim 11 , wherein the decoded sequence is generated autoregressively per token at each step via a self-attention module, a cross-attention module and a feed-forward module.
17 . The system of claim 11 , wherein the decoded sequence is generated by the decoder in a form of a conditional probability distribution of the decoded sequence conditioned on the global input representation.
18 . The system of claim 17 , wherein the operations further comprise:
computing a loss objective based on an entropy of the conditional probability distribution of the decoded sequence conditioned on the global input representation; and
updating parameters of the encoder and the decoder by minimizing the loss objective.
19 . The system of claim 11 , wherein the answer is generated by parsing the decoded sequence based on an answer indicator.
20 . A non-transitory processor-readable storage medium storing a plurality of processor-executable instructions for multi-hop question answering and reasoning via a natural language processing (NLP) model, the instructions being executed by a processor to perform operations comprising:
receiving, via a communication interface, a multi-hop question and a collection of passages;
generating a plurality of input blocks, each of which contains a concatenation of the multi-hop question, a respective title of a respective passage, and a respective context representation of the respective passage;
encoding, via an encoder, the plurality of input blocks into a plurality of encoded input representations;
concatenating the plurality of encoded input representations into a global input representation;
generating, via a decoder in response to the global input representation, a decoded sequence containing a title block, a supporting fact block and an answer block; and
generating an answer to the multi-hop question based on the answer block and a reasoning path accompanying the answer based on the title block and the supporting fact block.