IP Library Granted Patent US 8,738,617
Granted Patent B2
US 8,738,617 · App. 13/242,447 · Granted May 27, 2014

Providing answers to questions using multiple models to score candidate answers

Inventors: Eric W. Brown (New Fairfield, CT); David A. Ferrucci (Yorktown Heights, NY); James W. Murdock, IV (Millwood, NY)
Assignee: International Business Machines Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,738,617
App. No.
13/242,447
Granted
May 27, 2014
Kind
B2
Abstract

A method, system and computer program product for generating answers to questions. In one embodiment, the method comprises receiving an input query; conducting a search to identify candidate answers to the input query, and producing a plurality of scores for each of the candidate answers. For each of the candidate answers, one, of a plurality of candidate ranking functions, is selected. This selected ranking function is applied to the each of the candidate answers to determine a ranking for the candidate answer based on the scores for that candidate answer. One or more of the candidate answers is selected, based on the rankings for the candidate answers, as one or more answers to the input query. In an embodiment, the ranking function selection is performed using information about the question. In an embodiment, the ranking function selection is performed using information about each answer.

Claims (59)

1. A system for generating answers to questions, comprising:

a computer device comprising:

at least one distinct software module, each distinct software module being embodied on a tangible computer-readable medium;

a memory device storing a plurality of candidate scoring models; and

at least one processor coupled to the memory device and operative for:

receiving an input query;

using a candidate answer generation module to conduct a search, based on said query, in one or more data sources to identify candidate answers to the input query;

using a candidate answer scoring module to determine a plurality of first scores for each of the candidate answers;

for each of the candidate answers,

selecting one of the candidate scoring models based on defined criteria to determine a second, classification score for said each candidate answer; and

for each of the candidate answers, outputting the plurality of first scores determined for the candidate answer from the candidate answer scoring module, transmitting said first scores to the selected one of the candidate scoring models, and using said selected one of the candidate scoring models to process said first scores for said each candidate answer to determine said second, classification score for said each candidate answer; and

selecting one or more of the candidate answers, based on the second, classification scores for the candidate answers, as one or more answers to the input query.

2. The system according to claim 1 , wherein said selected one of the candidate scoring functions is selected based on the input query.

3. The system according to claim 2 , wherein:

the conducting a search includes extracting defined information about the input query; and

the selected one of the candidate scoring functions is selected based on said defined information.

4. The system according to claim 1 , wherein the selected one of the candidate scoring functions is selected based on said each candidate answer.

5. The system according to claim 4 , wherein:

the producing a plurality of scores for each of the candidate answers includes extracting defined information about each of the candidate answers; and

the selected one of the candidate scoring functions is selected based on said extracted defined information.

6. The system according to claim 1 , wherein the selecting one of the candidate scoring models includes:

determining a defined measure of a grammatical complexity of the input query; and

using the measure of the grammatical complexity of the input query to select one of the candidate scoring models to determine the second classification scores for all of the candidate answers identified for the input query.

7. An article of manufacture, comprising:

at least one tangible computer readable device having computer readable program code logic tangibly embodied therein to generate answers to questions, and for use with a memory device storing a plurality of candidate scoring models, the computer readable program code logic, when executing, performing the following:

receiving an input query;

using a candidate answer generation module to conduct a search, based on said query, in one or more data sources to identify candidate answers to the input query;

using a candidate answer scoring module to determine a plurality of first scores for each of the candidate answers;

for each of the candidate answers,

selecting one of the candidate scoring models based on defined criteria to determine a second, classification score for said each candidate answer; and

for each of the candidate answers, outputting the plurality of first scores determined for the candidate answer from the candidate answer scoring module, transmitting said first scores to the selected one of the candidate scoring models, and using said selected one of the candidate scoring models to process said first scores for said each candidate answer to determine said second, classification score for said each candidate answer; and

selecting one or more of the candidate answers, based on the second, classification scores for the candidate answers, as one or more answers to the input query.

8. The article of manufacture according to claim 7 , wherein said selected one of the candidate scoring functions is selected based on the input query.

9. The article of manufacture according to claim 7 , wherein:

the conducting a search includes extracting defined information about the input query; and

the selected one of the candidate scoring functions is selected based on said defined information.

10. The article of manufacture according to claim 7 , wherein the selected one of the candidate scoring functions is selected based on said each candidate answer.

11. The article of manufacture according to claim 10 , wherein:

the producing a plurality of scores for each of the candidate answers includes extracting defined information about each of the candidate answers; and

the selected one of the candidate scoring functions is selected based on said extracted defined information.

12. A system for generating answers to questions, comprising:

a computer device comprising:

at least one distinct software module, each distinct software module being embodied on a tangible computer-readable medium;

a memory device storing a plurality of candidate scoring models; and

at least one processor coupled to the memory device and operative for:

receiving a plurality of input queries;

for each query, using a candidate answer generation module to conduct a search, based on said each query, in one or more data sources to identify candidate answers to said each query;

using a candidate answer scoring module to determine a plurality of first scores for each of the candidate answers;

for each of the candidate answers,

selecting one of the candidate scoring functions based on defined criteria to determine a second, classification score for said each candidate answer; and

for each of the candidate answers, outputting the plurality of first scores determined for the candidate answer from the candidate answer scoring module, transmitting said first scores to the selected one of the candidate scoring functions, and using selected one of the candidate scoring functions to process said first scores for said each candidate answer to determine said second, classification score for said each candidate answer; and

selecting one or more of the candidate answers, based on the second, classification scores for the candidate answers, as one or more answers to the input query; and wherein:

for the candidate answers to each of a first group of the input queries, the selecting one of the candidate scoring functions includes selecting one of the candidate scoring functions based on the input query; and

for the candidate answers to each of a second group of the input queries, the selecting one of the candidate ranking functions includes selecting one of the candidate ranking functions based on said each candidate answer.

13. The system according to claim 12 , wherein:

defined information is extracted about the input query; and

the one of the candidate scoring functions selected based on the input query is selected based on said defined information;

defined information is extracted about each of the candidate answers; and

the one of the candidate scoring functions selected based on the candidate answers is selected based on said extracted defined information about the candidate answers.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 6, 2011
From: BROWN, ERIC W.; FERRUCCI, DAVID A.; MURDOCK, JAMES W., IV
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 027338/0761 →
Continuity (2)
Provisional Application 61387180 · Sep 28, 2010
Related Publication 20120078891A1 · Mar 29, 2012