Systems and methods for reading comprehension for a question answering task
Embodiments described herein provide a pipelined natural language question answering system that improves a BERT-based system. Specifically, the natural language question answering system uses a pipeline of neural networks each trained to perform a particular task. The context selection network identifies premium context from context for the question. The question type network identifies the natural language question as a yes, no, or span question and a yes or no answer to the natural language question when the question is a yes or no question. The span extraction model determines an answer span to the natural language question when the question is a span question.
1. A system comprising:
at least one memory including a pipeline neural network;
a processor coupled to the at least one memory; and
the pipeline neural network configured to:
receive an input text of a natural language question and context that provides an answer to the natural language question, wherein the context includes a plurality of paragraphs;
select, using a context selection network in the pipeline neural network, a premium context from the context, wherein the premium context includes at least two paragraphs;
determine, using a question type network in the pipeline neural network, that the natural language question is answered by an answer span; and
determine, using a span extraction network in the pipeline neural network and the premium context, the answer span as the answer to the natural language question.
2. The system of claim 1 , wherein the context selection network includes a classifier neural network configured to classify a first portion of the context as premium context and a second portion of the context as non-premium context.
3. The system of claim 1 , wherein the premium context includes at least one paragraph having a probability above a probability threshold.
4. The system of claim 1 , wherein the question type network includes a three-classifier neural network and configured to classify a type of the natural language question as a yes question, a no question, or a span question.
5. The system of claim 1 , wherein the context selection network includes a Bidirectional Encoder Representations from Transformers (BERT) model and a classification layer and is further configured to train the classification layer to select the premium context from the context.
6. The system of claim 1 , wherein the question type network includes a Bidirectional Encoder Representations from Transformers (BERT) model and a classification layer and is further configured to train the classification layer to determine a type of the answer to the natural language question.
7. The system of claim 1 , wherein the span extraction network includes a Bidirectional Encoder Representations from Transformers (BERT) model that is trained using a question-paragraph dataset to determine the answer span for the answer.
8. The system of claim 1 , wherein the pipeline neural network is further configured to divide the natural language question and the context into a plurality of tokens as inputs to the context selection network.
9. A system comprising:
at least one memory including a pipeline neural network;
a processor coupled to the at least one memory; and
the pipeline neural network configured to:
receive an input text of a natural language question and context that provides an answer to the natural language question, wherein the context includes a plurality of paragraphs;
select, using a context selection network in the pipeline neural network, a premium context from the context, wherein the premium context includes at least one paragraph having a probability above a probability threshold;
determine, using a question type network in the pipeline neural network, that the natural language question is answered by an answer span; and
determine, using a span extraction network in the pipeline neural network and the premium context, the answer span as the answer to the natural language question.
10. The system of claim 9 , wherein the context selection network includes a classifier neural network configured to classify a first portion of the context as premium context and a second portion of the context as non-premium context.
11. The system of claim 9 , wherein the question type network includes a three-classifier neural network and configured to classify a type of the natural language question as a yes question, a no question, or a span question.
12. The system of claim 9 , wherein the context selection network includes a Bidirectional Encoder Representations from Transformers (BERT) model and a classification layer and is further configured to train the classification layer to select the premium context from the context.
13. The system of claim 9 , wherein the question type network includes a Bidirectional Encoder Representations from Transformers (BERT) model and a classification layer and is further configured to train the classification layer to determine a type of the answer to the natural language question.
14. The system of claim 9 , wherein the span extraction network includes a Bidirectional Encoder Representations from Transformers (BERT) model that is trained using a question-paragraph dataset to determine the answer span for the answer.
15. A system comprising:
at least one memory including a pipeline neural network;
a processor coupled to the at least one memory; and
the pipeline neural network configured to:
receive an input text of a natural language question and context that provides an answer to the natural language question;
select, using a context selection network in the pipeline neural network, a premium context from the context;
determine, using a question type network in the pipeline neural network, that the natural language question is answered by an answer span, wherein the question type network includes a three-classifier neural network and configured to classify a type of the natural language question as a yes question, a no question, or a span question; and
determine, using a span extraction network in the pipeline neural network and the premium context, the answer span as the answer to the natural language question.
16. The system of claim 15 , wherein the context selection network includes a classifier neural network configured to classify a first portion of the context as premium context and a second portion of the context as non-premium context.
17. The system of claim 15 , wherein the context includes a plurality of paragraphs and the premium context includes at least two paragraphs.
18. The system of claim 15 , wherein the context selection network includes a Bidirectional Encoder Representations from Transformers (BERT) model and a classification layer and is further configured to train the classification layer to select the premium context from the context.
19. The system of claim 15 , wherein the question type network includes a Bidirectional Encoder Representations from Transformers (BERT) model and a classification layer and is further configured to train the classification layer to determine a type of the answer to the natural language question.
20. The system of claim 15 , wherein the span extraction network includes a Bidirectional Encoder Representations from Transformers (BERT) model that is trained using a question-paragraph dataset to determine the answer span for the answer.