Systems and methods for reading comprehension for a question answering task
Embodiments described herein provide a pipelined natural language question answering system that improves a BERT-based system. Specifically, the natural language question answering system uses a pipeline of neural networks each trained to perform a particular task. The context selection network identifies premium context from context for the question. The question type network identifies the natural language question as a yes, no, or span question and a yes or no answer to the natural language question when the question is a yes or no question. The span extraction model determines an answer span to the natural language question when the question is a span question.
1. A method comprising:
receiving, at a pipeline neural network, an input text of a natural language question and context that provides an answer to the natural language question;
selecting, using a context selection network in the pipeline neural network, a premium context from the context, wherein the premium context includes at least two paragraphs;
determining, using a question type network in the pipeline neural network, a type of the natural language question, wherein the question type network includes a Bidirectional Encoder Representations from Transformers (BERT) model and a classification layer and further comprising training the classification layer for determining the type of the natural language question; and
determining the answer to the natural language question as a yes or no answer when the type of the natural language question is a yes or a no question.
2. The method of claim 1 , wherein a span extraction network includes a BERT model that is trained using a question-paragraph dataset to determine an answer span for the answer.
3. The method of claim 1 , wherein the context selection network includes a classifier neural network for classifying a first portion of the context as the premium context and a second portion of the context as a non-premium context.
4. The method of claim 1 , wherein the context includes a plurality of paragraphs and the premium context includes at least one paragraph in the plurality of paragraphs having a probability above a probability threshold.
5. The method of claim 1 , wherein the question type network includes a three-classifier neural network for classifying the type of the natural language question as the yes question, the no question, or the span question.
6. The method of claim 1 , wherein the context selection network includes a BERT model and a classification layer and further comprising training the classification layer for selecting the premium context from the context.
7. A method comprising:
receiving, at a pipeline neural network, an input text of a natural language question and context that provides an answer to the natural language question;
selecting, using a context selection network in the pipeline neural network, a premium context from the context, wherein the premium context includes at least one paragraph having a probability above a probability threshold;
determining, using a question type network in the pipeline neural network, a type of the natural language question, wherein the question type network includes a Bidirectional Encoder Representations from Transformers (BERT) model and a classification layer and further comprising training the classification layer for determining the type of the natural language question; and
determining the answer to the natural language question as a yes or no answer when the type of the natural language question is a yes or a no question.
8. The method of claim 7 , wherein a span extraction network includes a BERT model that is trained using a question-paragraph dataset to determine an answer span for the answer.
9. The method of claim 7 , wherein the context selection network includes a classifier neural network for classifying a first portion of the context as the premium context and a second portion of the context as a non-premium context.
10. The method of claim 7 , wherein the question type network includes a three-classifier neural network for classifying the type of the natural language question as the yes question, the no question, or the span question.
11. The method of claim 7 , wherein the context selection network includes a BERT model and a classification layer and further comprising training the classification layer for selecting the premium context from the context.
12. A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations that determine an answer to a natural language question, the operations comprising:
receiving, at a pipeline neural network, an input text of the natural language question and context that provides the answer to the natural language question;
selecting, using a context selection network in the pipeline neural network, a premium context from the context;
determining, using a question type network in the pipeline neural network, a type of the natural language question, wherein the question type network includes a three-classifier neural network for classifying the type of the natural language question as the yes question, the no question, or the span question; and
determining the answer to the natural language question as a yes or no answer when the type of the natural language question is a yes or a no question.
13. The non-transitory machine-readable medium of claim 12 , wherein the context selection network includes a classifier neural network for classifying a first portion of the context as the premium context and a second portion of the context as a non-premium context.
14. The non-transitory machine-readable medium of claim 12 , wherein the context includes a plurality of paragraphs and the premium context includes at least two paragraphs.
15. The non-transitory machine-readable medium of claim 12 , wherein the context includes a plurality of paragraphs and the premium context includes at least one paragraph in the plurality of paragraphs having a probability above a probability threshold.
16. The non-transitory machine-readable medium of claim 12 , wherein the context selection network includes a Bidirectional Encoder Representations from Transformers (BERT) model and a classification layer and further comprising training the classification layer for selecting the premium context from the context.
17. The non-transitory machine-readable medium of claim 12 , wherein the question type network includes a Bidirectional Encoder Representations from Transformers (BERT) model and a classification layer and further comprising training the classification layer for determining the type of the natural language question.
18. The non-transitory machine-readable medium of claim 12 , wherein a span extraction network includes a BERT model that is trained using a question-paragraph dataset to determine an answer span for the answer.