IP Library › Granted Patent US 9,805,718
Granted Patent B2
US 9,805,718 · App. 13/866,509 · Granted Oct 31, 2017

Clarifying natural language input using targeted questions

Inventors: Necip Fazil Ayan (Palo Alto, CA); Arindam Mandal (San Mateo, CA); Jing Zheng (San Jose, CA)
Assignee: SRI INTERNAITONAL
G10L15/22G06F17/2785G10L15/24H04M3/4936G10L15/1815G10L2015/225
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,805,718
App. No.
13/866,509
Granted
Oct 31, 2017
Kind
B2
Abstract

A dialog assistant embodied in a computing system can present a clarification question based on a machine-readable version of human-generated conversational natural language input. Some versions of the dialog assistant identify a clarification target in the machine-readable version, determine a clarification type relating to the clarification target, present the clarification question in a conversational natural language manner, and process a human-generated conversational natural language response to the clarification question.

Claims (119)

1. A computer-implemented method, comprising:

receiving, by a computing device, input, wherein the input is associated with an interaction with the computing device, and wherein the input includes dialog input, application input, or sensor input;

determining an interaction history, wherein determining includes adding the input to the interaction history, wherein adding includes using an artificial intelligence-based model to combine the input with prior interaction history;

determining a current context, wherein the current context is associated with an interaction with the computing device, and wherein the interaction includes dialog input, application input, or sensor input;

receiving verbal input, wherein the verbal input includes natural language, and wherein the verbal input includes a plurality of words;

interpreting the verbal input, wherein interpreting the verbal input includes assigning one or more attributes to the plurality of words;

identifying an attribute from the one or more attributes, wherein the identified attribute is associated with one or more words from the plurality of words, wherein the identified attribute indicates that the one or more words are unclear, and wherein a word is unclear when the computing system cannot recognize the word;

determining that the current context and the interaction history do not clarify the one or more words;

determining a type of information needed to recognize the one or more words, wherein determining includes using the one or more attributes assigned to the one or more words to identify missing information;

determining a clarification question for the one or more words, wherein determining includes using the type of information, the current context, and the interaction history, wherein the clarification question is formatted to request the missing information, and wherein the format is customized using the current context and the interaction history; and

outputting the clarification question.

2. The computer-implemented method of claim 1 , wherein application input includes data or instructions output by an application executing on the computing device.

3. The computer-implemented method of claim 1 , wherein the sensor input is generated automatically.

4. The computer-implemented method of claim 1 , further comprising:

receiving non-verbal input, wherein the non-verbal input includes a physical motion interpreted from a sensor associated with the computing device, and wherein the current context includes the non-verbal input.

5. The computer-implemented method of claim 1 , further comprising:

receiving additional verbal input;

determining that the additional verbal input includes the missing information.

6. The computer-implemented method of claim 1 , further comprising:

receiving additional verbal input;

filtering one or more non-responsive portions of the additional verbal input; and

extracting the missing information from the filtered additional verbal input.

7. The computer-implemented method of claim 1 , further comprising

receiving additional verbal input;

filtering one or more non-responsive portions of the additional verbal input; and

determining that no part of the filtered additional verbal input includes the missing information.

8. The computer-implemented method of claim 7 , further comprising:

determining a second clarification question, wherein the second clarification question is different from the clarification question for the one or more words; and

outputting the second clarification question.

9. The computer-implemented method of claim 1 , further comprising:

receiving additional verbal input;

using the additional verbal input, the current context, and the interaction history to determine an action; and

executing the action, wherein the action modifies an operation of the computing device.

10. The computer-implemented method of claim 9 , wherein using the additional verbal input includes modifying the verbal input to include the additional verbal input.

11. The computer-implemented method of claim 9 , wherein using the additional verbal input includes generating new verbal input, wherein the new verbal input combines the verbal input and the additional verbal input.

12. The computer-implemented method of claim 1 , wherein the interaction history includes a clarification history, wherein the clarification history includes one or more previous clarification questions, and wherein determining the clarification question further includes using the clarification history.

13. The computer-implemented method of claim 1 , further comprising:

determining one or more additional attributes from the one or more attributes, wherein the one or more attributes indicate that one or more sets of words from the plurality of words are unclear; and

determining an order for the identified attribute and the one or more additional attributes, wherein the order determines an order in which to clarify the identified attribute and the one or more attributes.

14. A computing device, comprising:

one or more processors; and

a non-transitory computer-readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including:

receiving input, wherein the input is associated with an interaction with the computing device, and wherein the input includes dialog input, application input, or sensor input;

determining an interaction history, wherein determining includes adding the input to the interaction history, wherein adding includes using an artificial intelligence-based model to combine the input with prior interaction history;

determining a current context, wherein the current context is associated with an interaction with the computing device, and wherein the interaction includes dialog input, application input, or sensor input;

receiving verbal input, wherein the verbal input includes natural language, and wherein the verbal input includes a plurality of words;

interpreting the verbal input, wherein interpreting the verbal input includes assigning one or more attributes to the plurality of words;

identifying an attribute from the one or more attributes, wherein the identified attribute is associated with one or more words from the plurality of words, wherein the identified attribute indicates that the one or more words are unclear, and wherein a word is unclear when the computing system cannot recognize the word;

determining that the current context and the interaction history do not clarify the one or more words;

determining a type of information needed to recognize the one or more words, wherein determining includes using the one or more attributes assigned to the one or more words to identify missing information;

determining a clarification question for the one or more words, wherein determining includes using the type of information, the current context, and the interaction history, wherein the clarification question is formatted to request the missing information, and wherein the format is customized using the current context and the interaction history; and

outputting the clarification question.

15. The computing device of claim 14 , wherein application input includes data or instructions output by an application executing on the computing device.

16. The computing device of claim 14 , wherein the sensor input is generated automatically.

17. The computing device of claim 14 , wherein the non-transitory computer-readable medium further includes instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including:

receiving non-verbal input, wherein the non-verbal input includes a physical motion interpreted from a sensor associated with the computing device, and wherein the current context includes the non-verbal input.

18. The computing device of claim 14 , wherein the non-transitory computer-readable medium further includes instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including:

receiving additional verbal input;

determining that the additional verbal input includes the missing information.

19. The computing device of claim 14 , wherein the non-transitory computer-readable medium further includes instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including:

receiving additional verbal input;

filtering one or more non-responsive portions of the additional verbal input; and

extracting the missing information from the filtered additional verbal input.

20. The computing device of claim 14 , wherein the non-transitory computer-readable medium further includes instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including:

receiving additional verbal input;

filtering one or more non-responsive portions of the additional verbal input; and

determining that no part of the filtered additional verbal input includes the missing information.

21. The computing device of claim 20 , wherein the non-transitory computer-readable medium further includes instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including:

determining a second clarification question, wherein the second clarification question is different from the clarification question for the one or more words; and

outputting the second clarification question.

22. The computing device of claim 14 , wherein the non-transitory computer-readable medium further includes instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including:

receiving additional verbal input;

using the additional verbal input, the current context, and the interaction history to determine an action; and

executing the action, wherein the action modifies an operation of the computing device.

23. The computing device of claim 22 , wherein using the additional verbal input includes modifying the verbal input to include the additional verbal input.

24. The computing device of claim 22 , wherein using the additional verbal input includes generating new verbal input, wherein the new verbal input combines the verbal input and the additional verbal input.

25. The computing device of claim 14 , wherein the interaction history includes a clarification history, wherein the clarification history includes one or more previous clarification questions, and wherein determining the clarification question further includes using the clarification history.

26. The computing device of claim 14 , wherein the non-transitory computer-readable medium further includes instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including:

determining one or more additional attributes from the one or more attributes, wherein the one or more attributes indicate that one or more sets of words from the plurality of words are unclear; and

determining an order for the identified attribute and the one or more additional attributes, wherein the order determines an order in which to clarify the identified attribute and the one or more attributes.

27. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions that, when executed by one or more processors, cause the one or more processors to:

receive input, wherein the input is associated with an interaction with a computing device, and wherein the input includes dialog input, application input, or sensor input;

determine an interaction history, wherein determining includes adding the input to the interaction history, wherein adding includes using an artificial intelligence-based model to combine the input with prior interaction history;

determine a current context, wherein the current context is associated with an interaction with the computing device, and wherein the interaction includes dialog input, application input, or sensor input;

receive verbal input, wherein the verbal input includes natural language, and wherein the verbal input includes a plurality of words;

interpret the verbal input, wherein interpreting the verbal input includes assigning one or more attributes to the plurality of words;

identify an attribute from the one or more attributes, wherein the identified attribute is associated with one or more words from the plurality of words, wherein the identified attribute indicates that the one or more words are unclear, and wherein a word is unclear when the computing system cannot recognize the word;

determine that the current context and the interaction history do not clarify the one or more words;

determine a type of information needed to recognize the one or more words, wherein determining includes using the one or more attributes assigned to the one or more words to identify missing information;

determine a clarification question for the one or more words, wherein determining includes using the type of information, the current context, and the interaction history, wherein the clarification question is formatted to request the missing information, and wherein the format is customized using the current context and the interaction history; and

output the clarification question.

28. The computer-program product of claim 27 , wherein application input includes data or instructions output by an application executing on the computing device.

29. The computer-program product of claim 27 , wherein the sensor input is generated automatically.

30. The computer-program product of claim 27 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

receive non-verbal input, wherein the non-verbal input includes a physical motion interpreted from a sensor associated with the computing device, and wherein the current context includes the non-verbal input.

31. The computer-program product of claim 27 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

receive additional verbal input;

determine that the additional verbal input includes the missing information.

32. The computer-program product of claim 27 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

receive additional verbal input;

filter one or more non-responsive portions of the additional verbal input; and

extract the missing information from the filtered additional verbal input.

33. The computer-program product of claim 27 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

receive additional verbal input;

filter one or more non-responsive portions of the additional verbal input; and

determine that no part of the filtered additional verbal input includes the missing information.

34. The computer-program product of claim 33 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

determine a second clarification question, wherein the second clarification question is different from the clarification question for the one or more words; and

output the second clarification question.

35. The computer-program product of claim 27 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

receive additional verbal input;

use the additional verbal input, the current context, and the interaction history to determine an action; and

execute the action, wherein the action modifies an operation of the computing device.

36. The computer-program product of claim 35 , wherein using the additional verbal input includes modifying the verbal input to include the additional verbal input.

37. The computer-program product of claim 35 , wherein using the additional verbal input includes generating new verbal input, wherein the new verbal input combines the verbal input and the additional verbal input.

38. The computer-program product of claim 27 , wherein the interaction history includes a clarification history, wherein the clarification history includes one or more previous clarification questions, and wherein determining the clarification question further includes using the clarification history.

39. The computer-program product of claim 27 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

determine one or more additional attributes from the one or more attributes, wherein the one or more attributes indicate that one or more sets of words from the plurality of words are unclear; and

determine an order for the identified attribute and the one or more additional attributes, wherein the order determines an order in which to clarify the identified attribute and the one or more attributes.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 10, 2013
From: AYAN, NECIP FAZIL; MANDAL, ARINDAM; ZHENG, JING
To: SRI INTERNATIONAL
Reel/Frame 030396/0156 →
Continuity (1)
Related Publication 20140316764A1 · Oct 23, 2014