Lexical answer type confidence estimation and application
A system, method and computer program product for automatically estimating the confidence of a detected LAT to provide a more accurate overall score for an obtained candidate answer. A confidence “score” or value of each detected LAT is obtained, and the system and method performs combining the confidence score with a degree of match between a LAT and an AnswerType of the candidate answer to provide improved overall score for the candidate answer.
1. A computer-implemented method for automatically generating answers to questions comprising the steps of:
determining a lexical answer type (LAT) associated with an input query;
computing a first score for said input query LAT, wherein said first score is indicative of a quality of said query LAT;
obtaining a candidate answer to said input query from an information source;
determining an answer type (AT) for said candidate answer;
comparing said query LAT with said candidate answer AT;
computing a second score representing a degree of match between said compared query LAT with said candidate answer AT; and
combining said first score and said second score to provide a total score indicative of a quality of said candidate answer,
wherein a processing device automatically performs one or more of said determining a query LAT, computing said first score and second score, obtaining candidate answers, determining a AT, comparing and combining.
2. The computer-implemented method of claim 1 , wherein said computing a first score for said query LAT associated with said input query comprises:
extracting a set of features describing said determined query LAT; and,
applying a model to the feature set to produce a LAT confidence value representing an estimated accuracy of the determined query LAT, said first score comprising said confidence value.
3. The computer-implemented method of claim 2 , wherein said applied model is one or more of: a logistic regression model or a decision tree model constructed manually or automatically from prior sets of training instances.
4. The computer-implemented method of claim 2 , wherein a feature of said extracted feature set comprises a pattern rule, said extracting a set of features comprising:
identifying a syntactic pattern rule associated with said query, said pattern rule identified from a set of pattern rules that define common LATs.
5. The computer-implemented method of claim 2 , wherein a feature of said extracted feature set comprises a frequency of a query word being a candidate LAT, said extracting a set of features further comprising:
checking prior instances of a detected LAT word; and
computing a LAT word frequency based on said prior instances.
6. The computer-implemented method of claim 2 , wherein a feature of said extracted feature set comprises a part of speech of the candidate LAT, said extracting a set of features further comprising:
obtaining a parse tree data structure associated with said query; and
traversing said parse tree to identify grammatical relations amongst words associated with said candidate LAT.
7. The computer-implemented method of claim 2 , wherein a feature of said extracted feature set comprises a co-reference information, said extracting a set of features further comprising: determining whether the candidate LAT word co-references some other word in said query recognized as a LAT.
8. The computer-implemented method of claim 2 , wherein one or more features of said extracted feature set are extracted in parallel.
9. The computer-implemented method of claim 2 , wherein said computing a second score representing a degree of match between said compared query LAT with said candidate answer AT includes:
matching said candidate answer against instances in a corpus or a knowledge base;
retrieving a type associated with an instance in the corpus or the knowledge base; and,
matching said query LAT with said retrieved type to produce a type coercion (TyCor) value, said second score comprising said TyCor value.
10. The computer-implemented method of claim 9 , wherein a query has up to n candidate LATs, said combining to provide said total score is a summation according to:
(confidence score LAT1 )(TyCor LAT 1 )+(confidence score LAT2 )(TyCor LAT 2 )+ . . . +(confidence score LATn )(TyCor LAT n )
wherein confidence score LAT1 , confidence score LAT2 , and confidence score LATn are produced confidence values representing estimated accuracies of each determined LAT of said query, and TyCor LAT1 , TyCor LAT2 , and TyCor LATn are the produced type coercion (TyCor) values of a candidate answer with each LAT.