IP Library Granted Patent US 11,355,115
Granted Patent B2
US 11,355,115 · App. 16/878,907 · Granted Jun 7, 2022

Question answering for a voice user interface

Inventors: William Folwell Barton (Concord, MA); Raluca Teodora Stoian (Cambridge, MA); James Alfred Cuff (Medway, MA); Matthew Aaron Cember (Seattle, WA)
Assignee: Amazon Technologies, Inc.
G10L15/22G10L13/00G10L15/063G10L15/1815G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,355,115
App. No.
16/878,907
Granted
Jun 7, 2022
Kind
B2
Abstract

Techniques for receiving user input and soliciting responses from one or more users are described. A user may speak a question to a system. The system may determine that various question and answering content sources cannot provide an answer to the question. The system may then store data representing the question in a dedicated storage. The system may a topic, to which the question relates and provide the question to users having knowledge on the topic.

Claims (103)

1. A method, comprising:

receiving, from a first device, first data corresponding to a first user input;

determining the first user input corresponds to a question;

processing the first data with respect to at least one knowledge source to determine that an answer to the question is unknown;

determining a user identifier (ID) corresponding to a first user that provided the first user input;

storing, in a storage, an association between the user ID and the question;

after storing the association, determining a first answer to the question;

causing the first device to present a notification representing an answer to the question has been determined;

after causing the first device to present the notification, receiving, from the first device, second data corresponding to a second user input;

determining first content responsive to the second user input;

determining the second user input corresponds to a topic;

determining the question corresponds to the topic;

causing the first device to present the first content in response to the second user input; and

based at least in part on determining the second user input and the question both correspond to the topic, causing the first device to present the first answer in response to the second user input.

2. The method of claim 1 , further comprising:

performing text-to-speech (TTS) processing to generate audio data including synthesized speech corresponding to the first answer,

wherein causing the first device to present the first answer comprises causing the first device to output the audio data.

3. The method of claim 1 , further comprising:

determining the second data corresponds to a first user interaction received from the first device on a calendar day; and

causing the first device to present the first answer further based at least in part on the second data corresponding to the first user interaction received from the first device on the calendar day.

4. The method of claim 1 , wherein the first data is audio data, wherein the first user input is an utterance, and wherein the method further comprises:

determining first audio characteristics representing the audio data; and

determining the first audio characteristics correspond to stored audio characteristics associated with the user ID.

5. The method of claim 1 , further comprising:

generating a vector representation of the first data; and

storing the association between the user ID and the vector representation.

6. The method of claim 1 , wherein processing the first data to determine an answer to the question is unknown comprises:

processing the first data with respect to the at least one knowledge source to determine a second answer to the question;

determining the second answer is associated with a first confidence score; and

determining the first confidence score is below a threshold confidence score.

7. The method of claim 1 , further comprising:

presenting a plurality of text, including a portion corresponding to the question, to a second user via a second device; and

receiving, from the second device, third data representing the first answer.

8. A method comprising:

receiving, from a first device, first data corresponding a first user input;

determining the first user input corresponds to a question;

processing the first data with respect to at least one knowledge source to determine that an answer to the question is unknown;

determining a user identifier (ID) corresponding to a user that provided the first user input;

storing, in a storage, an association between the user ID and the question;

after storing the association, determining a first answer to the question;

causing the first device to present a notification representing an answer to the question has been determined;

after causing the first device to present the notification, receiving, from the first device, second data corresponding to a second user input;

determining first content responsive to the second user input;

determining the second data corresponds to a first user interaction received from the first device on a calendar day;

causing the first device to present the first content in response to the second user input; and

based at least in part on the second data corresponding to the first user interaction, causing the first device to present the first answer in response to the second user input.

9. A computing system, comprising:

at least one processor; and

at least one memory comprising instructions that, when executed by the at least one processor, cause the computing system to:

receive, from a first device, first data corresponding to a first user input;

determine the first user input corresponds to a question;

process the first data with respect to at least one knowledge source to determine that an answer to the question is unknown;

determine a user identifier (ID) corresponding to a user that provided the first user input;

store, in a storage, an association between the user ID and the question;

after storing the association, determine a first answer to the question;

cause the first device to present a notification representing an answer to the question has been determined;

after causing the first device to present the notification, receive, from the first device, second data corresponding to a second user input;

determine first content responsive to the second user input;

determine the second user input corresponds to a topic;

determine the question corresponds to the topic;

cause the first device to present the first content in response to the second user input; and

based at least in part on determining the second user input and the question both correspond to the topic, cause the first device to present the first answer in response to the second user input.

10. The computing system of claim 9 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:

perform text-to-speech (TTS) processing to generate audio data including synthesized speech corresponding to the first answer,

wherein the instructions that cause the computing system to cause the first device to present the first answer further comprise cause the first device to output the audio data.

11. The computing system of claim 9 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:

determine the second data corresponds to a first user interaction received from the first device on a calendar day; and

cause the first device to present the first answer further based at least in part on the second data corresponding to the first user interaction received from the first device on the calendar day.

12. The computing system of claim 9 , wherein the first data is audio data, wherein the first user input is an utterance, and wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:

determine first audio characteristics representing the audio data; and

determine the first audio characteristics correspond to stored audio characteristics associated with the user ID.

13. The computing system of claim 9 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:

generate a vector representation of the first data; and

store the association between the user ID and the vector representation.

14. The computing system of claim 9 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:

present a plurality of text, including a portion correspond to the question, to a second user via a second device; and

receive, from the second device, third data representing the first answer.

15. The method of claim 1 , further comprising:

performing natural language understanding (NLU) processing on the first data to generate NLU result data comprising an intent representing the first user input; and

storing the association between the user ID and the NLU results data.

16. The computing system of claim 9 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the computing system to:

perform natural language understanding (NLU) processing on the first data to generate NLU result data comprising an intent representing the first user input; and

store the association between the user ID and the NLU results data.

17. The computing system of claim 9 , wherein the instructions that cause the computing system to process the first data to determine an answer to the question is unknown further comprise instructions to:

process the first data with respect to the at least one knowledge source to determine a second answer to the question;

determine the second answer is associated with a first confidence score; and

determining the first confidence score is below a threshold confidence score.

18. The method of claim 8 , wherein the first data is audio data, wherein the first user input is an utterance, and wherein the method further comprises:

determining first audio characteristics representing the audio data;

determining the first audio characteristics correspond to stored audio characteristics associated with the user ID;

determining user profile data associated with the user ID;

determining the user profile data represents an answer to a previous unanswerable question is to be output with respect to a first user interaction, associated with the user ID, on a calendar day; and

causing the first device to present the first answer further based at least in part on the user profile data representing an answer to a previously unanswerable question is to be output with respect to a first user interaction, associated with the user ID, on a calendar day.

19. The method of claim 8 , further comprising:

receiving, from the first device, third data corresponding to a third user input;

determining second content responsive to the third user input, the second content comprising at least a first word;

determining the first answer corresponds to the first word;

causing the first device to present the second content; and

causing the first device to present the first answer further based at least in part on determining the first answer corresponds to the first word.

20. The method of claim 8 , wherein processing the first data to determine an answer to the question is unknown further comprises:

processing the first data with respect to the at least one knowledge source to determine a second answer;

determining the second answer is associated with a first confidence score; and

determining the first confidence score is below a threshold confidence score.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 20, 2020
From: BARTON, WILLIAM FOLWELL; STOIAN, RALUCA TEODORA; CUFF, JAMES ALFRED; CEMBER, MATTHEW AARON
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 052712/0065 →
Continuity (3)
Continuation 15911987 · Mar 5, 2018
Provisional Application 62616988 · Jan 12, 2018
Related Publication 20200286481A1 · Sep 10, 2020