Intelligent evidence classification and notification in a deep question answering system
System, method, and computer program product to identify changes in evidence used to answer questions by a deep question answering system, by identifying a first evidence related to a feature, the deep question answering system having identified the feature as being relevant to answering a first type of question of a plurality of types of questions, and responsive to: (i) detecting a change in the first evidence, and (ii) determining that a confidence score of a first response generated for a first question, of the first type of question, exceeds a confidence threshold, generating an updated response for the first question based on the changed first evidence.
1. A method to identify changes in evidence used to answer questions by a question answering system, comprising:
identifying a first evidence related to a feature, wherein the question answering system has identified the feature as being relevant to answering a first type of question of a plurality of types of questions;
detecting a change in the first evidence
determining that a confidence score of a first response generated for a first question, of the first type of question, exceeds a confidence threshold;
outputting an indication of the change in the first evidence; and
responsive to receiving an approval to update the first response, generating an updated response for the first question.
2. The method of claim 1 , wherein a user provides the approval.
3. The method of claim 1 , wherein the updated response is based on at least one of: (i) processing of a second question, of the first type, and (ii) reprocessing the first question.
4. The method of claim 3 , wherein the second question is generated by one of: (i) the question answering system, and (ii) a user.
5. The method of claim 1 , wherein the first evidence is unavailable, wherein the change in the first evidence comprises an availability of the first evidence.
6. The method of claim 1 , wherein the first evidence is available, wherein the change in the first evidence comprises a change in a content of the first evidence.
7. The method of claim 1 , further comprising monitoring the first evidence to detect the change.
8. The method of claim 7 , wherein the first evidence is monitored upon determining that a relevance score of the first evidence exceeds a relevance threshold, wherein the first evidence comprises a content of at least one of: (i) a document, (ii) a variable, (iii) an article, and (iv) a web page, and wherein the first question is part of a first case comprising at least the first question.
9. A method to identify changes in evidence used to answer questions by a question answering system, the method comprising:
identifying a first evidence related to a feature, wherein the question answering system has identified the feature as being relevant to answering a first type of question of a plurality of types of questions;
generating an updated response for the first question based on the changed first evidence responsive to: (i) detecting a change in the first evidence, and (ii) determining that a confidence score of a first response generated for a first question, of the first type of question, exceeds a confidence threshold; and
notifying a user of the change in the first evidence, wherein the updated response is generated upon approval of the user.
10. The method of claim 9 , further comprising monitoring the first evidence to detect the change.
11. The method of claim 10 , wherein the first evidence is monitored upon determining that a relevance score of the first evidence exceeds a relevance threshold, wherein the first evidence comprises a content of at least one of: (i) a document, (ii) a variable, (iii) an article, and (iv) a web page, and wherein the first question is part of a first case comprising at least the first question.