IP Library › Granted Patent US 11,100,922
Granted Patent B1
US 11,100,922 · App. 15/716,477 · Granted Aug 24, 2021

System and methods for triggering sequences of operations based on voice commands

Inventors: Rohan Mutagi (Redmond, WA); Vibhav Salgaonkar (Seattle, WA); Philip Lee (Seattle, WA); Bo Li (Kenmore, WA); Vibhu Gavini (Mercer Island, WA)
Assignee: Amazon Technologies, Inc.
G10L15/22G10L13/00G10L15/1815G10L17/22H04L12/2816G10L15/14G10L2015/088G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,100,922
App. No.
15/716,477
Filed
Sep 26, 2017
Granted
Aug 24, 2021
Kind
B1
Art Unit
2659
USPC
704/235
Abstract

This disclosure is directed to systems, methods, and devices related to providing the execution of multi-operation sequences based on a trigger occurring which may be a voice-controlled utterance or execution may be based on a trigger occurring and a condition occurring. In accordance with various principles disclosed herein, multi-operation sequences may be executed based on voice-controlled commands and the identification that a trigger has occurred. The voice-controlled electronic devices can be configured to communicate with, and to directly control the operation of, a wide array of other devices. These devices can include, without limitation, outlets that can be turned ON and OFF remotely such that anything plugged into them can be controlled, turning lights ON and OFF, setting the temperature of a network accessible thermostat, etc.

Claims (83)

1. A computer-implemented method comprising:

during a first time period:

receiving first input data from a first device;

determining the first input data corresponds to a trigger to perform a plurality of operations;

receiving stored data associated with the trigger, the stored data indicating execution of the plurality of operations occurs when a time condition is satisfied;

determining first time information associated with the first input data; and

determining the time condition is unsatisfied based at least in part on the first time information; and

during a second time period after the first time period:

receiving first data indicating the time condition is satisfied:

determining the trigger was previously determined; and

causing the plurality of operations to be executed based at least in part on receiving the first data and determining the trigger was previously determined.

2. The computer-implemented method of claim 1 , wherein receiving the first input data comprises receiving audio data corresponding to an utterance, and the computer-implemented method further comprises:

performing automatic speech recognition (ASR) processing on the audio data to generate ASR output data; and

performing natural language processing on the ASR output data to determine an intent corresponding to the trigger.

3. The computer-implemented method of claim 1 , further comprising:

receiving the first data from an event bus, the first data representing a present time; and

determining the present time satisfies the time condition.

4. The computer-implemented method of claim 1 , further comprising:

determining the time condition corresponds to at least one of a first day of week or a first time of day; and

determining the time condition is unsatisfied based at least in part on determining the first time information corresponds to at least one of a second day of week or a second time of day.

5. The computer-implemented method of claim 1 , further comprising:

determining a sequence of execution of the plurality of operations, the sequence indicating that a first operation of the plurality of operations is to be performed before a second operation of the plurality of operations,

wherein causing the plurality of operations to executed comprises causing the first operation to be executed before the second operation.

6. The computer-implemented method of claim 1 , wherein causing the plurality of operations to be executed comprises:

generating a command indicating an intent and at least one entity.

7. A computer-implemented method comprising:

during a first time period:

receiving, from a first device, audio data corresponding to an utterance;

performing speech processing on the audio data to determine the utterance corresponds to a trigger to perform a plurality of operations;

receiving stored data associated with the trigger, the stored data indicating execution of the plurality of operations occurs when a time condition is satisfied;

determining first time information associated with the audio data; and

determining the time condition is unsatisfied based at least in part on the first time information; and

during a second time period after the first time period:

receiving first data indicating the time condition is satisfied;

determining the trigger was previously determined; and

causing the plurality of operations to be executed based at least in part on receiving the first data and determining the trigger was previously determined.

8. The computer-implemented method of claim 7 , further comprising:

receiving the first data from an event bus, the first data representing a present time; and

determining the present time satisfies the time condition.

9. The computer-implemented method of claim 7 , further comprising:

determining the time condition corresponds to at least one of a first day of week or a first time of day; and

determining the time condition is unsatisfied based at least in part on determining the first time information corresponds to at least one of a second day of week or a second time of day.

10. The computer-implemented method of claim 7 , wherein performing the speech processing comprises:

performing automatic speech recognition (ASR) processing on the audio data to generate ASR output data; and

performing natural language processing on the ASR output data to determine an intent corresponding to the trigger.

11. The computer-implemented method of claim 7 , further comprising:

determining a sequence of execution of the plurality of operations, the sequence indicating that a first operation of the plurality of operations is to be performed before a second operation of the plurality of operations,

wherein causing the plurality of operations to executed comprises causing the first operation to be executed before the second operation.

12. The computer-implemented method of claim 7 , wherein causing the plurality of operations to be executed comprises:

generating a command indicating an intent and at least one entity.

13. The computer-implemented method of claim 7 , wherein causing the plurality of operations to be executed comprises:

generating a smart home command indicating an intent and at least one entity, the smart home command directing a second device to perform at least a first operation; and

sending the smart home command to an event bus.

14. A system comprising:

memory; and

at least one processor operable to:

during a first time period:

receive, from a first device, audio data corresponding to an utterance;

performing speech processing on the audio data to determine the utterance corresponds to a trigger to perform a plurality of operations;

receive stored data associated with the trigger, the stored data indicating execution of the plurality of operations occurs when a time condition is satisfied;

determine first time information associated with the audio data; and

determine the time condition is unsatisfied based at least in part on the first time information; and

during a second time period after the first time period:

receive first data indicating the time condition is satisfied;

determine the trigger was previous determined; and

cause the plurality of operations to be executed based at least in part on receiving the first data and determining the trigger was previously determined.

15. The system of claim 14 , wherein the at least one processor is further operable to:

receive the first data from an event bus, the first data representing, a present time; and

determine the present time satisfies the time condition.

16. The system of claim 14 , wherein the at least one processor is further operable to:

determine the time condition corresponds to at least one of a day of week or a time of day; and

determine the time condition is unsatisfied based at least in part on determining the first time information corresponds to at least one of a second day of week or a second time of day.

17. The system of claim 14 , wherein the at least one processor is further operable to:

perform automatic speech recognition (ASR) processing on the audio data to generate ASR output data; and

perform natural language processing on the ASR output data to determine an intent corresponding to the trigger.

18. The system of claim 14 , wherein the at least one processor is further operable to:

determine a sequence of execution of the plurality of operations, the sequence indicating that a first operation of the plurality of operations is to be performed before a second operation of the plurality of operations; and

cause the first operation to be executed before the second operation.

19. The system of claim 14 , wherein the at least one processor is further operable to:

cause the plurality of operations to be executed by generating a command indicating an intent and at least one entity.

20. The system of claim 14 , wherein the at least one processor is further operable to:

generate a smart home command indicating an intent and at least one entity, the smart home command directing a second device to perform at least a first operation; and

send the smart home command to an event bus.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 11, 2021
From: SALGAONKAR, VIBHAV
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 056195/0547 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 12, 2017
From: MUTAGI, ROHAN; SALGAOKAR, VIBHAV; LEE, PHILIP; LI, BO; GAVINI, VIBHU
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 043853/0293 →
Cited By (15)
US 12,190,879 US 12,223,226 US 12,243,522 US 12,251,214 US 12,315,615 US 12,327,067 US 12,343,137 US 12,354,744 US 12,367,877 US 12,389,330 US 12,468,557 US 12,502,125 US 12,633,407 US 12,718,940 US 12,749,486