Extracting explanations from supporting evidence
A method trains an inference model on two-hop NLI problems that include a first and second premise and a hypothesis, and further includes generating, by the model using hypothesis reduction, an explanation from an input premise and an input hypothesis, for an input single hop NLI problem. The learning step determines a distribution over extraction starting positions and lengths from within the first premise and hypothesis of a two-hop NLI problem. The learning step k extraction output slots with combinations of words from the first premise of the two-hop NLI problem and fills another extraction output slots with combinations of words from the hypothesis of the two-hop NLI problem. The learning step trains a sequence model by using the extraction output slots and the other extraction output slots together with the second premise as an input to a single-hop NLI classifier to output a label of the two-hop NLI problem.
1. A computer-implemented method for extracting explanations, comprising:
learning hypothesis reduction by training a feed forward network model on two-hop Natural Language Inference (NLI) problems that include a first premise, a second premise, and a hypothesis;
generating, by the trained feed forward network model using hypothesis reduction, an explanation in a form of a selection of words from an input premise and an input hypothesis, for an input single hop NLI problem to determine a connection status of a connected computer system that includes the input premise and the input hypothesis;
controlling a hardware switch to transfer connection to a secure network within the connected computer system based on the explanation;
wherein said learning step comprises:
determining, using the feed forward network model, a distribution over extraction starting positions and extraction lengths from within the first premise and the hypothesis of a two-hop NLI problem from among the two-hop NLI problems;
filling, responsive to the distribution, a first k extraction output slots with combinations of words from the first premise of the two-hop NLI problem;
filling, responsive to the distribution, a second k extraction output slots with combinations of words from the hypothesis of the two-hop NLI problem;
training the feed forward network model by using the first k extraction output slots and the second k extraction output slots together with the second premise as an input to a single-hop NLI classifier to output a label of the two-hop NLI problem by:
applying hypothesis reduction to a hypothesis and a first premise from a training set of two-hop NLI problem to obtain a second hypothesis;
applying the trained single hop NLI classifier to the second premise and the second hypothesis;
computing a final loss from a trained single hope NLI classifier and regularization penalty of a hypothesis reduction module of the feed forward network model; and
backpropagating the final loss through the feed forward network model.
2. The computer-implemented method of claim 1 , where a regularization penalty is added to discourage the separate extractions of overlapping spans of words from among the combinations of words from the first premise and the hypothesis of the two-hop NLI problem.
3. The computer-implemented method of claim 1 , wherein a final loss is specified by a sum of a classification loss and a term formed by multiplying a sum of a premise regularization loss and a hypothesis regularization loss by a tunable hyperparameter.
4. The computer-implemented method of claim 3 , wherein the premise regularization loss and the hypothesis regularization loss are sums of products of weights in the k extraction output slots and the second k extraction output slots.
5. The computer-implemented method of claim 1 , wherein said generating step selects words by taking the words with a largest contribution to each of the extraction output slots.
6. The computer-implemented method of claim 1 , wherein said generating step selects words whose contribution to each of the extraction output slots exceeds a threshold.
7. The computer-implemented method of claim 1 , where weights of the single hop NLI classifier are not updated as the feed forward network model is trained.
8. The computer-implemented method of claim 1 , wherein two-hop NLI problems with an entailment label are constructed from two-hop question answering problems by taking a question and the answer to the question to form the input hypothesis, taking the first premise to be an evidence sentence that includes the answer, and taking the second premise to be the evidence sentence that does not include the answer.
9. The computer-implemented method of claim 1 , wherein two-hop NLI problems with a non-entailment label are constructed from two-hop question answering problems by taking a question and the answer to the question to form the input hypothesis, taking the first premise to be an evidence sentence that includes the answer, and selecting a non-evidence sentence from an article with the same title as the first premise to be the second premise.
10. The computer-implemented method of claim 1 , wherein two-hop NLI problems with a non-entailment label are constructed from two-hop question answering problems by taking a question and the answer to the question to form the input hypothesis, taking the first premise to be an evidence sentence that includes the answer, and taking the second premise to be an evidence sentence that does not include the answer, modified by replacing its named entities by randomly chosen named entities of a same type.
11. A computer program product for extracting explanations, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
learning hypothesis reduction by training an feed forward network model on two-hop Natural Language Inference (NLI) problems that include a first premise, a second premise, and a hypothesis;
generating, by the trained feed forward network model using hypothesis reduction, an explanation in a form of a selection of words from an input premise and an input hypothesis, for an input single hop NLI problem to determine a connection status of a connected computer system that includes the input premise and the input hypothesis;
controlling a hardware switch to transfer connection to a secure network within the connected computer system based on the explanation;
wherein said learning step comprises:
determining, using the feed forward network model, a distribution over extraction starting positions and extraction lengths from within the first premise and the hypothesis of a two-hop NLI problem from among the two-hop NLI problems;
filling, responsive to the distribution, a first k extraction output slots with combinations of words from the first premise of the two-hop NLI problem;
filling, responsive to the distribution, a second k extraction output slots with combinations of words from the hypothesis of the two-hop NLI problem;
training the feed forward network model by using the first k extraction output slots and the second k extraction output slots together with the second premise as an input to a single-hop NLI classifier to output a label of the two-hop NLI problem by:
applying hypothesis reduction to a hypothesis and a first premise from a training set of two-hop NLI problem to obtain a second hypothesis;
applying the trained single hop NLI classifier to the second premise and the second hypothesis;
computing a final loss from a trained single hope NLI classifier and regularization penalty of a hypothesis reduction module of the feed forward network model; and
backpropagating the final loss through the feed forward network model.
12. The computer program product of claim 11 , where a regularization penalty is added to discourage the separate extractions of overlapping spans of words from among the combinations of words from the first premise and the hypothesis of the two-hop NLI problem.
13. The computer program product of claim 11 , wherein a final loss is specified by a sum of a classification loss and a term formed by multiplying a sum of a premise regularization loss and a hypothesis regularization loss by a tunable hyperparameter.
14. The computer program product of claim 13 , wherein the premise regularization loss and the hypothesis regularization loss are sums of products of weights in the k extraction output slots and the second k extraction output slots.
15. The computer program product of claim 11 , wherein said generating step selects words by taking the words with a largest contribution to each of the extraction output slots.
16. The computer program product of claim 11 , wherein said generating step selects words whose contribution to each of the extraction output slots exceeds a threshold.
17. The computer program product of claim 11 , where weights of the single hop NLI classifier are not updated as the feed forward network model is trained.
18. The computer program product of claim 11 , wherein two-hop NLI problems with an entailment label are constructed from two-hop question answering problems by taking a question and the answer to the question to form the input hypothesis, taking the first premise to be an evidence sentence that includes the answer, and taking the second premise to be the evidence sentence that does not include the answer.
19. A computer-implemented method for extracting explanations, comprising:
a memory device for storing program code; and
a processor device, operatively coupled to the memory device, for running the program code to
learn hypothesis reduction by training a feed forward network model on two-hop Natural Language Inference (NLI) problems that include a first premise, a second premise, and a hypothesis;
generate, by the trained feed forward network model using hypothesis reduction, an explanation in a form of a selection of words from an input premise and an input hypothesis, for an input single hop NLI problem to determine a connection status of a connected computer system that includes the input premise and the input hypothesis;
control a hardware switch to transfer connection to a secure network within the connected computer system based on the explanation;
wherein the processor device learns the hypothesis reduction by
determining, using the feed forward network model, a distribution over extraction starting positions and extraction lengths from within the first premise and the hypothesis of a two-hop NLI problem from among the two-hop NLI problems;
filling, responsive to the distribution, a first k extraction output slots with combinations of words from the first premise of the two-hop NLI problem;
filling, responsive to the distribution, a second k extraction output slots with combinations of words from the hypothesis of the two-hop NLI problem;
training the feed forward network model by using the first k extraction output slots and the second k extraction output slots together with the second premise as an input to a single-hop NLI classifier to output a label of the two-hop NLI problem by:
applying hypothesis reduction to a hypothesis and a first premise from a training set of two-hop NLI problem to obtain a second hypothesis;
applying the trained single hop NLI classifier to the second premise and the second hypothesis;
computing a final loss from a trained single hope NLI classifier and regularization penalty of a hypothesis reduction module of the feed forward network model; and
backpropagating the final loss through the feed forward network model.