IP Library Granted Patent US 8,832,064
Granted Patent B2
US 8,832,064 · App. 11/319,188 · Granted Sep 9, 2014

Answer determination for natural language questioning

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,832,064
App. No.
11/319,188
Granted
Sep 9, 2014
Kind
B2
Abstract

Open-domain question answering is the task of finding a concise answer to a natural language question using a large domain, such as the Internet. The use of a semantic role labeling approach to the extraction of the answers to an open domain factoid (Who/When/What/Where) natural language question that contains a predicate is described. Semantic role labeling identities predicates and semantic argument phrases in the natural language question and the candidate sentences. When searching for an answer to a natural language question, the missing argument in the question is matched using semantic parses of the candidate answers. Such a technique may improve the accuracy of a question answering system and may decrease the length of answers for enabling voice interface to a question answering system.

Claims (47)

1. A method comprising:

receiving a natural language question from a user;

generating a search phrase by:

applying an exact phrase technique which retrieves an exact phrase from the received natural language question; and

applying a conjunction technique which identifies a sub-phrase from the natural language question and conjoins the sub-phrase with the exact phrase as a predicate, to yield a search phrase that comprises a plurality of words in a non-stop order;

identifying a plurality of candidate sentences based on the search phrase, wherein each candidate sentence in the plurality of candidate sentences contains the plurality of words in the non-stop order;

evaluating the plurality of candidate sentences using a 3-tier cascaded approach, wherein the 3-tier cascaded approach utilizes a baseline approach in combination with a semantic role labeler approach, where the baseline approach is a first tier of the 3-tier cascaded approach, the semantic role labeler approach is a second tier of the 3-tier cascaded approach, and a conjunction of sub-phrase search which is a mixture of the baseline approach and the semantic role labeler approach is a third tier of the 3-tier cascaded approach, to yield a level of precision for each candidate sentence;

when the level of precision of a candidate sentence meets a required threshold, generating a candidate answer by using a phrase sentence extraction approach to extract a portion from a side of the search phrase within the candidate sentence; and

providing the candidate answer in response to the natural language question.

2. The method of claim 1 , wherein identifying of the candidate sentences further comprises:

scanning a plurality of documents for the search phrase.

3. The method of claim 1 , wherein the candidate answer is one of a plurality of candidate answers, and further comprising ranking each candidate answer in the plurality of candidate answers by a confidence level.

4. The method of claim 3 , wherein the candidate answer has a highest confidence level in the plurality of candidate answers.

5. The method of claim 1 , wherein the conjunction of the sub-phrase with the exact phrase further comprises determining whether a predicate and a search argument are found within the natural language question.

6. The method of claim 5 , wherein the search argument is classified as a location argument.

7. The method of claim 5 , wherein the search argument is classified as a temporal argument.

8. The method of claim 1 , wherein the natural language question comprises one of who, when, where, and what words.

9. The method of claim 1 , wherein the exact phrase technique is based on dropping all words between a “wh” word and an auxiliary verb, comprising the “wh” word and the auxiliary verb, and matching a tense of a predicate verb in the natural language question with the auxiliary verb.

10. The method of claim 9 , wherein the auxiliary verb comprises one of: is, was, are, did, do and does.

11. A computer-readable storage device having instructions stored which, when executed by a computing device, cause the computing device to perform operations comprising:

receiving a natural language question from a user;

generating a search phrase by:

applying an exact phrase technique which retrieves an exact phrase from the received natural language question; and

applying a conjunction technique which identifies a sub-phrase from the natural language question and conjoins the sub-phrase with the exact phrase as a predicate, to yield a search phrase that comprises a plurality of words in a non-stop order;

identifying a plurality of candidate sentences based on the search phrase, wherein each candidate sentence in the plurality of candidate sentences contains the plurality of words in the non-stop order;

evaluating the plurality of candidate sentences using a 3-tier cascaded approach, wherein the 3-tier cascaded approach utilizes a baseline approach in combination with a semantic role labeler approach, where the baseline approach is a first tier of the 3-tier cascaded approach, the semantic role labeler approach is a second tier of the 3-tier cascaded approach, and a conjunction of sub-phrase search which is a mixture of the baseline approach and the semantic role labeler approach is a third tier of the 3-tier cascaded approach, to yield a level of precision for each candidate sentence;

when the level of precision of a candidate sentence meets a required threshold, generating a candidate answer by using a phrase sentence extraction approach to extract a portion from a side of the search phrase within the candidate sentence; and

providing the candidate answer in response to the natural language question.

12. The computer-readable storage device of claim 11 , wherein the natural language question comprises one of who, when, where, and what words.

13. The computer-readable storage device of claim 12 , wherein when the natural language question contains an auxiliary verb, the search phrase is generated by dropping words between the auxiliary verb and the one of: who, when, where, and what words.

14. The computer-readable storage device of claim 13 , wherein the auxiliary verb comprises one of: is, was, are, did, do, and does.

15. The computer-readable storage device of claim 11 , wherein determining the candidate sentence comprises:

scanning a plurality of documents for an occurrence of the search phrase within a sentence.

16. A system comprising:

a processor; and

a memory having instructions stored which, when executed by the processor, cause the processor to perform operations comprising:

receiving a natural language question from a user;

generating a search phrase by:

applying an exact phrase technique which retrieves an exact phrase from the received natural language question; and

applying a conjunction technique which identifies a sub-phrase from the natural language question and conjoins the sub-phrase with the exact phrase as a predicate, to yield a search phrase that comprises a plurality of words in a non-stop order;

identifying a plurality of candidate sentences based on the search phrase, wherein each candidate sentence in the plurality of candidate sentences contains the plurality of words in the non-stop order;

evaluating the plurality of candidate sentences using a 3-tier cascaded approach, wherein the 3-tier cascaded approach utilizes a baseline approach in combination with a semantic role labeler approach, where the baseline approach is a first tier of the 3-tier cascaded approach, the semantic role labeler approach is a second tier of the 3-tier cascaded approach, and a conjunction of sub-phrase search which is a mixture of the baseline approach and the semantic role labeler approach is a third tier of the 3-tier cascaded approach, to yield a level of precision for each candidate sentence;

when the level of precision of a candidate sentence meets a required threshold, generating a candidate answer by using a phrase sentence extraction approach to extract a portion from a side of the search phrase within the candidate sentence; and

providing the candidate answer in response to the natural language question.

17. The system of claim 16 , wherein identifying of the candidate sentences further comprises scanning a plurality of documents for the search phrase within a sentence and identifying a candidate document within a plurality of documents based on which document contains the search phrase in a sentence.

18. The system of claim 16 , wherein the candidate document is obtained by searching a fixed document set for the candidate document.

19. The system of claim 16 , wherein the natural language question comprises one of who, when, where, and what words.

Assignments (7)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 13, 2023
From: NUANCE COMMUNICATIONS, INC.
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 065552/0934 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 26, 2017
From: AT&T INTELLECTUAL PROPERTY II, L.P.
To: NUANCE COMMUNICATIONS, INC.
Reel/Frame 041512/0608 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNOR NAME PREVIOUSLY RECORDED ON REEL 033956 FRAME 0367. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 18, 2016
From: AT&T PROPERTIES, LLC
To: AT&T INTELLECTUAL PROPERTY II, L.P.
Reel/Frame 039735/0334 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME PREVIOUSLY RECORDED AT REEL: 017616 FRAME: 0923. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT . Recorded Aug 18, 2016
From: STENCHIKOVA, SVETLANA; HAKKANI-TUR, DILEK; TUR, GOKHAN
To: AT&T CORP.
Reel/Frame 039736/0733 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2016
From: AT&T CORP.
To: AT&T PROPERTIES, LLC
Reel/Frame 039476/0790 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 15, 2014
From: STENCHIKOVA, SVETLANA; HAKKANI-TUR, DILEK; TUR, GOKHAN
To: AT&T INTELLECTUAL PROPERTY II, L.P.
Reel/Frame 033956/0367 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 15, 2006
From: STENCHIKOVA, SVETLANA; TUR, DILEK HAKKANI; TUR, GOKHAN
To: AT&T CORPORATION
Reel/Frame 017616/0923 →