IP Library Granted Patent US 8,600,986
Granted Patent B2
US 8,600,986 · App. 13/597,795 · Granted Dec 3, 2013

Lexical answer type confidence estimation and application

Inventors: James J. Fan (Hawthorne, NY); David A. Ferrucci (Yorktown Heights, NY); David C. Gondek (Hawthorne, NY); Aditya A. Kalyanpur (Westwood, NJ); Adam P. Lally (Cold Spring, NY); James W. Murdock (Millwood, NY); Wlodek W. Zadrozny (Tarrytown, NY)
Assignee: International Business Machines Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,600,986
App. No.
13/597,795
Granted
Dec 3, 2013
Kind
B2
Abstract

A system, method and computer program product for automatically estimating the confidence of a detected LAT to provide a more accurate overall score for an obtained candidate answer. A confidence “score” or value of each detected LAT is obtained, and the system and method performs combining the confidence score with a degree of match between a LAT and an AnswerType of the candidate answer to provide improved overall score for the candidate answer.

Claims (45)

1. A system for automatically generating answers to questions comprising:

a memory storage device;

one or more processor devices, each in communication to said memory device and configured for performing a method comprising:

determining a lexical answer type (LAT) associated with an input query;

computing a first score for said query LAT, wherein said first score is indicative of a quality of said query LAT;

obtaining a candidate answer to said input query from an information source;

determining an answer type (AT) for said candidate answer;

comparing said query LAT with said candidate answer AT;

computing a second score representing a degree of match between said compared query LAT with said candidate answer AT; and

combining said first score and said second score to provide a total score indicative of a quality of said candidate answer.

2. The system of claim 1 , wherein said one or more processor devices for computing a first score for said query LAT is configure to carry out said computing step by:

extracting a set of features describing said determined query LAT; and,

applying a model to the feature set to produce a LAT confidence value representing an estimated accuracy of the determined query LAT, said first score comprising said confidence value.

3. The system of claim 2 , wherein said one or more processor devices are configured to apply one or more of: a logistic regression model or a decision tree model constructed manually or automatically from prior sets of training instances.

4. The system of claim 2 , wherein a feature of said extracted feature set comprises a pattern rule, said one or more processor devices for extracting a set of features further performs:

identifying a syntactic pattern rule associated with said query, said pattern rule identified from a set of pattern rules that define common LATs.

5. The system of claim 2 , wherein a feature of said extracted feature set comprises a frequency of a query word being a candidate LAT, said one or more processor devices for extracting a set of features further performs:

checking prior instances of a detected LAT word; and

computing a LAT word frequency based on said prior instances.

6. The system of claim 2 , wherein a feature of said extracted feature set comprises a part of speech of the candidate LAT, said one or more processor devices for extracting a set of features further performs:

obtaining a parse tree data structure associated with said query; and

traversing said parse tree to identify grammatical relations amongst words associated with said candidate LAT.

7. The system of claim 2 , wherein a feature of said extracted feature set comprises a co-reference information, said one or more processor devices for extracting a set of features further performs: determining whether the candidate LAT word co-references some other word in said query recognized as a LAT.

8. The system of claim 2 , wherein said one or more processor devices for computing a second score is configure to carry out said computing step by:

matching said candidate answer against instances in a corpus or a knowledge base;

retrieving a type associated with an instance in the corpus or the knowledge base; and,

matching said query LAT with said received type to produce a type coercion (TyCor) value, said second score comprising said TyCor value.

9. The system of claim 8 , wherein a query has n candidate LATs, said one or more processor devices for combining to provide said total score performs a summation according to:

(confidence score LAT1 )(TyCor LAT1 )+(confidence score LAT2 )(TyCor LAT2 )+ . . . +(confidence score LATn )(TyCor LATn )

wherein confidence score LAT1 , confidence score LAT2 , and confidence score LATn are produced confidence values representing estimated accuracies of each determined query LAT, and TyCor LAT1 , TyCor LAT2 , and TyCor LATn are the produced type coercion (TyCor) values of a candidate answer with each LAT.

10. A computer program product for automatically generating answers to questions, the computer program device comprising a storage medium readable by a processing circuit and storing instructions run by the processing circuit for performing a method, the method comprising:

determining a lexical answer type (LAT) associated with an input query; computing a first score for said query LAT, wherein said first score is indicative of a quality of said query LAT;

obtaining a candidate answer to said input query from an information source;

determining an answer type (AT) for said candidate answer;

comparing said query LAT with said candidate answer AT;

computing a second score representing a degree of match between the compared query LAT with said candidate answer AT; and

combining said first score and said second score to provide a total score indicative of a quality of said candidate answer.

11. The computer program product for automatically generating answers to questions of claim 10 , wherein said computing a first score for each LAT associated with said input query comprises:

extracting a set of features describing said computed query LAT; and,

applying a model to the feature set to produce a LAT confidence value representing an estimated accuracy of the determined query LAT, said first score comprising said LAT confidence value.

12. The computer program product for automatically generating answers to questions of claim 11 , wherein:

a first feature of said extracted feature set comprises a pattern rule, said extracting a set of features comprising identifying a syntactic pattern rule associated with said query, said pattern rule identified from a set of pattern rules that define common LATs; and wherein,

a second feature of said extracted feature set comprises a frequency of a query word being a candidate LAT, said extracting a set of features further comprising checking prior instances of a detected LAT word and computing a LAT word frequency based on said prior instances; and wherein,

a third feature of said extracted feature set comprises a part of speech of the candidate LAT, said extracting a set of features further comprising obtaining a parse tree data structure associated with said query; and traversing said parse tree to identify grammatical relations amongst words associated with said candidate LAT; and, wherein,

a fourth feature of said extracted feature set comprises a co-reference information, said extracting a set of features further comprising determining whether the candidate LAT word co-references some other word in said query recognized as a LAT.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 26, 2022
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: HYUNDAI MOTOR COMPANY; KIA CORPORATION
Reel/Frame 058780/0252 →
Continuity (3)
Continuation 13242290 · Sep 23, 2011
Provisional Application 61386029 · Sep 24, 2010
Related Publication 20120323906A1 · Dec 20, 2012