Passage justification scoring for question answering

1. A system, comprising:

a memory having computer readable computer instructions; and

a processor for executing the computer readable instructions, the computer readable instructions including:

creating a multi-layered neural network from domain knowledge;

training the multi-layered neural network with labeled data and unlabeled data;

inputting at least one of an existing passage justification component and raw input data for a question and passage to the trained multi-layered neural network;

extracting, via the trained multi-layered neural network and from results of inputting the at least one existing passage justification component and raw input data for a question and passage, concepts determined to have passage justification;

creating, via machine learning, a passage justification model from the extracted concepts and from passage justification; and

scoring, via the passage justification model, the extracted concepts.

2. The system of claim 1 , wherein the existing passage justification components include at least one of:

focus-answer type matching features;

question-passage term matching features;

question-passage parse matching features;

question-passage dependency path matching features;

question-passage relation matching features; and

question-passage topic matching features.

3. The system of claim 1 , wherein the raw input data include at least one of:

bag of words features for the question;

bag of words features for the passage;

typing features for the question;

typing features for the passage;

topic features for the question;

topic features for the passage;

Ngram features for the question; and

Ngram features for the passage.

4. The system of claim 1 , wherein the computer readable instructions further include generating the labeled data and the unlabeled data via:

at least one of distant supervision using question-answer pairs and existing knowledge bases; and

full supervision with manually annotated data.

5. The system of claim 1 , wherein training the multi-layered neural network comprises using the labeled data with the multi-layered neural network to force the output of the multi-layered neural network to match corresponding labels of the labeled data.

6. The system of claim 1 , wherein training the multi-layered neural network comprises using the unlabeled data with the multi-layered neural network to minimize data reconstruction errors.

7. The system of claim 1 , wherein the extracting concepts determined to have passage justification includes using outputs from any selected one of the layers of the multi-layered neural network as concepts for input to a higher layer of the multi-layered neural network.

8. The system of claim 1 , wherein the multi-layered neural network includes at least one of a convolutional neural network and a deep neural network.

9. The system of claim 1 , wherein the multi-layered neural network includes a combination of stacked neural networks.

10. A computer program product comprising:

a non-transitory storage medium readable by a processor and storing instructions for execution by the processor to perform a method comprising:

creating a multi-layered neural network from domain knowledge;

training the multi-layered neural network with labeled data and unlabeled data;

inputting at least one of an existing passage justification component and raw input data for a question and passage to the trained multi-layered neural network;

extracting via the trained multi-layered neural network and from results of inputting the at least one existing passage justification component and raw input data for a question and passage concepts determined to have passage justification;

creating, via machine learning, a passage justification model from the extracted concepts and from passage justification; and

scoring, via the passage justification model, the extracted concepts.

11. The computer program product of claim 10 , wherein the extracting concepts determined to have passage justification includes using outputs from any selected one of the layers of the multi-layered neural network as concepts for input to a higher layer of the multi-layered neural network.