IP Library Granted Patent US 9,690,774
Granted Patent B1
US 9,690,774 · App. 14/971,331 · Granted Jun 27, 2017

Identifying vague questions in a question-answer system

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,690,774
App. No.
14/971,331
Granted
Jun 27, 2017
Kind
B1
Abstract

An approach is provided that improves a question answering (QA) computer system by reducing a number of vague questions submitted to the QA system. When a question is submitted to the QA system, the approach performs a vagueness question analysis on the question. The vagueness question analysis results in a vagueness score. The question is submitted to the QA system in response to the vagueness score reaching a threshold value that indicates a lack of vagueness in the question. The approach inhibits submission of the question to the QA system in response to the vagueness score failing to reach the threshold value.

Claims (82)

1. A method, implemented by an information handling system that includes a memory and a processor, that improves a question answering (QA) system by reducing a number of vague questions submitted to the QA system, the method comprising:

receiving a question that is submitted to the QA system;

performing a vagueness question analysis on the question, wherein the vagueness question analysis results in a vagueness score, and wherein the analysis further:

increases the vagueness score based on a set of linguistic features including a prepositional phrase in the question;

increases the vagueness score based on the set of linguistic features including a firm superlative in the question;

decreases the vagueness score based on the set of linguistic features including an arguable superlative in the question; and

decreases the vagueness score based on the set of linguistic features including a referential pronoun in the question;

submitting the question to the QA system in response to the vagueness score reaching a threshold value that indicates a lack of vagueness in the question; and

inhibiting submission of the question to the QA system in response to the vagueness score failing to reach the threshold value.

2. The method of claim 1 , wherein the performing the vagueness question analysis further comprises:

analyzing the question using natural language processing (NLP) that discovers the set of linguistic features pertaining to the question.

3. The method of claim 2 , further comprising:

providing a set of feedback to a requestor of the question, wherein the set of feedback is based on one or more of the linguistic features discovered in the question.

4. The method of claim 3 , further comprising:

in response to the inhibiting the submission of the question to the QA system:

providing one or more suggestions to improve the vagueness score of the question based on the linguistic features discovered in the question; and

prompting the requestor to rephrase the question in light of the provided one or more suggestions.

5. The method of claim 2 , wherein a higher vagueness score indicates a lower vagueness level of the question, wherein the method further comprises:

increasing the vagueness score based on the set of linguistic features including at least one concept in the question; and

increasing the vagueness score based on the set of linguistic features including at least one concept instance in the question.

6. The method of claim 1 , further comprising:

configuring the vagueness question analysis prior to receiving the question, wherein the configuring comprises:

setting the threshold value;

setting a first value to use when increasing the vagueness score based on the set of linguistic features including at least one concept in the question;

setting a second value to use when increasing the vagueness score based on the set of linguistic features including at least one concept instance in the question;

setting a third value to use when increasing the vagueness score based on the set of linguistic features including the prepositional phrase in the question;

setting a fourth value to use when increasing the vagueness score based on the set of linguistic features including the firm superlative in the question;

setting a fifth value to use when decreasing the vagueness score based on the set of linguistic features including the arguable superlative in the question; and

setting a sixth value to use when decreasing the vagueness score based on the set of linguistic features including the referential pronoun in the question.

7. An information handling system comprising:

one or more hardware processors;

one or more data stores accessible by at least one of the one or more hardware processors;

a hardware memory coupled to at least one of the one or more hardware processors; and

a set of computer program instructions stored in the hardware memory and executed by at least one of the one or more hardware processors in order to improve a question answering (QA) system by reducing a number of vague questions submitted to the QA system by performing actions of:

receiving a question that is submitted to the QA system;

performing a vagueness question analysis on the question, wherein the vagueness question analysis results in a vagueness score, and wherein the analysis further:

increases the vagueness score based on a set of linguistic features including a prepositional phrase in the question;

increases the vagueness score based on the set of linguistic features including a firm superlative in the question;

decreases the vagueness score based on the set of linguistic features including an arguable superlative in the question; and

decreases the vagueness score based on the set of linguistic features including a referential pronoun in the question;

submitting the question to the QA system in response to the vagueness score reaching a threshold value that indicates a lack of vagueness in the question; and

inhibiting submission of the question to the QA system in response to the vagueness score failing to reach the threshold value.

8. The information handling system of claim 7 , wherein the performing the vagueness question analysis further comprises:

analyzing the question using natural language processing (NLP) that discovers the set of linguistic features pertaining to the question.

9. The information handling system of claim 8 , further comprising:

providing a set of feedback to a requestor of the question, wherein the set of feedback is based on one or more of the linguistic features discovered in the question.

10. The information handling system of claim 9 , further comprising:

in response to the inhibiting the submission of the question to the QA system:

providing one or more suggestions to improve the vagueness score of the question based on the linguistic features discovered in the question; and

prompting the requestor to rephrase the question in light of the provided one or more suggestions.

11. The information handling system of claim 8 , wherein a higher vagueness score indicates a lower vagueness level of the question, wherein the method further comprises:

increasing the vagueness score based on the set of linguistic features including at least one concept in the question; and

increasing the vagueness score based on the set of linguistic features including at least one concept instance in the question.

12. The information handling system of claim 7 , further comprising:

configuring the vagueness question analysis prior to receiving the question, wherein the configuring comprises:

setting the threshold value;

setting a first value to use when increasing the vagueness score based on the set of linguistic features including at least one concept in the question;

setting a second value to use when increasing the vagueness score based on the set of linguistic features including at least one concept instance in the question;

setting a third value to use when increasing the vagueness score based on the set of linguistic features including the prepositional phrase in the question;

setting a fourth value to use when increasing the vagueness score based on the set of linguistic features including the firm superlative in the question;

setting a fifth value to use when decreasing the vagueness score based on the set of linguistic features including the arguable superlative in the question; and

setting a sixth value to use when decreasing the vagueness score based on the set of linguistic features including a referential pronoun in the question.

13. A computer program product stored in a computer readable storage medium, comprising computer program code that, when executed by an information handling system, causes the information handling system to improve a question answering (QA) computer system by reducing a number of vague questions submitted to the QA system by performing actions comprising:

receiving a question that is submitted to the QA system;

performing a vagueness question analysis on the question, wherein the vagueness question analysis results in a vagueness score, and wherein the analysis further:

increases the vagueness score based on a set of linguistic features including a prepositional phrase in the question;

increases the vagueness score based on the set of linguistic features including a firm superlative in the question;

decreases the vagueness score based on the set of linguistic features including an arguable superlative in the question; and

decreases the vagueness score based on the set of linguistic features including a referential pronoun in the question;

submitting the question to the QA system in response to the vagueness score reaching a threshold value that indicates a lack of vagueness in the question; and

inhibiting submission of the question to the QA system in response to the vagueness score failing to reach the threshold value.

14. The computer program product of claim 13 , wherein the performing the vagueness question analysis further comprises:

analyzing the question using natural language processing (NLP) that discovers the set of linguistic features pertaining to the question.

15. The computer program product of claim 14 , further comprising:

providing a set of feedback to a requestor of the question, wherein the set of feedback is based on one or more of the linguistic features discovered in the question.

16. The computer program product of claim 15 , further comprising:

in response to the inhibiting the submission of the question to the QA system:

providing one or more suggestions to improve the vagueness score of the question based on the linguistic features discovered in the question; and

prompting the requestor to rephrase the question in light of the provided one or more suggestions.

17. The computer program product of claim 14 , wherein a higher vagueness score indicates a lower vagueness level of the question, wherein the method further comprises:

increasing the vagueness score based on the set of linguistic features including at least one concept in the question; and

increasing the vagueness score based on the set of linguistic features including at least one concept instance in the question.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 15, 2021
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: AIRBNB, INC.
Reel/Frame 056427/0193 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2015
From: BEASON, WILLIAM A.; DOWLING, VINCENT J.; GATTIKER, ANNE E.; KRISHNAMURTHY, LAKSHMINARAYANAN; KOZHAYA, JOSEPH N.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 037307/0443 →