IP Library › Granted Patent US 11,380,330
Granted Patent B2
US 11,380,330 · App. 16/997,024 · Granted Jul 5, 2022

Conversational recovery for voice user interface

Inventors: Eliav Samuel Zimmern Kahan (Jamaica Plain, MA); Gregory Newell (Somerville, MA); Mahesh Guruswamy (Brookline, MA); Daren Gill (Concord, MA); Prashant Rao (Woburn, MA)
Assignee: Amazon Technologies, Inc.
G10L15/26G10L15/1815G06F16/4387G10L15/1822G10L15/32
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,380,330
App. No.
16/997,024
Granted
Jul 5, 2022
Kind
B2
Abstract

A processing device executing a component of a conversational recovery system receives an intent data and a first entity data identified from user input data. The processing device determines that the first entity data is associated with first content associated with a first component. The processing device additionally receives a text data of the user input data. The processing device determines a word in the text data that matches a keyword associated with second content associated with a second component. The processing device ranks the first component and the second component. The processing device generates message data that comprises an inquiry with respect to choosing the first content or the second content.

Claims (54)

1. A computer-implemented method comprising:

receiving input data representing a natural language input to a client device;

processing the input data to determine first natural language understanding (NLU) data representing a first interpretation of the natural language input;

determining the first NLU data corresponds to a potential failure response;

based at least in part on determining the first NLU data corresponds to a potential failure response, determining first data representing a semantic similarity between the input data and previous input data representing a previous natural language input;

determining, based at least in part on the first data, second NLU data corresponding to the previous input data and representing a second interpretation of the natural language input;

determining output data using the second NLU data; and

sending the output data to the client device.

2. The computer-implemented method of claim 1 , wherein processing the input data comprises:

performing NLU processing to determine the first NLU data and the second NLU data.

3. The computer-implemented method of claim 1 , further comprising:

performing NLU processing to determine the first NLU data and a first score corresponding to the first NLU data; and

prior to determining the first NLU data corresponds to a potential failure response, determining the first score corresponds to a preferred rank among a plurality of interpretations of the natural language input.

4. The computer-implemented method of claim 1 , wherein the first NLU data includes first intent data different than second intent data included in the second NLU data.

5. The computer-implemented method of claim 4 , wherein the first NLU data and the second NLU data includes first entity data.

6. The computer-implemented method of claim 1 , wherein:

the first NLU data corresponds to first content associated with a first component;

the method further comprises determining a portion of the input data corresponds to a second component; and

determining the first NLU data corresponds to a potential failure response is based at least in part on the portion of the input data corresponding to the second component.

7. The computer-implemented method of claim 1 , wherein determining the output data using the second NLU data is based at least in part on feedback data.

8. The computer-implemented method of claim 1 , further comprising:

determining the natural language input lacks a verb,

wherein determining the first NLU data corresponds to a potential failure response is based at least in part on the natural language input lacking a verb.

9. The computer-implemented method of claim 1 , wherein determining the output data using the second NLU data is performed at least in part to avoid presenting a failure response.

10. A system comprising:

at least one processor; and

at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:

receive input data representing a natural language input to a client device;

process the input data to determine a first interpretation of the natural language input;

determine the first interpretation corresponds to a potential failure response;

send, to the client device, first data requesting selection of a second interpretation of the natural language input, the second interpretation being different from the first interpretation;

determine output data based at least in part on the second interpretation; and

send the output data to the client device.

11. The system of claim 10 , wherein the first data further includes an indication of the first interpretation.

12. The system of claim 10 , wherein the input data comprises input audio data and wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

perform automatic speech recognition processing using the input audio data to determine automatic speech recognition (ASR) data;

perform text-to-speech processing using the ASR data to determine output audio data; and

include in the first data the output audio data.

13. The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

process the ASR data to determine natural language understanding data representing the first interpretation.

14. The system of claim 10 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine second data representing a semantic similarity between the natural language input and previous user input data corresponding to the second interpretation,

wherein the output data is determined based at least in part on the second data.

15. The system of claim 10 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine the natural language input lacks a verb,

wherein the instructions that cause the system to determine the first interpretation corresponds to a potential failure response are based at least in part on the natural language input lacking a verb.

16. The system of claim 10 , wherein the instructions that cause the system to determine the output data based at least in part on the second interpretation are performed based at least in part on feedback data.

17. The system of claim 10 , wherein the instructions that cause the system to process the input data comprise instructions that, when executed by the at least one processor, further cause the system to:

performing natural language understanding (NLU) processing to determine first NLU data corresponding to the first interpretation and second NLU data corresponding to the second interpretation.

18. The system of claim 17 , wherein the instructions that determine the first interpretation corresponds to a potential failure response are based at least in part on the first NLU data.

19. The system of claim 10 , wherein:

the first interpretation corresponds to first content associated with a first component;

the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to determine a portion of the input data corresponds to a second component; and

determine the first interpretation corresponds to a potential failure response based at least in part on the portion of the input data corresponding to the second component.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 19, 2020
From: KAHAN, ELIAV SAMUEL ZIMMERN; NEWELL, GREGORY; GURUSWAMY, MAHESH; GILL, DAREN; RAO, PRASHANT
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 053535/0722 →
Continuity (3)
Continuation 15842456 · Dec 14, 2017
Provisional Application 62582784 · Nov 7, 2017
Related Publication 20210027785A1 · Jan 28, 2021
Cited By (2)
US 12,190,870 US 12,586,574