Granted Patent B1

US 11,295,743 · App. 16/883,379 · Granted Apr 5, 2022

Inventors: Fabian Andreas Bumberger (Toronto, CA); Sabria Farheen (Toronto, CA); Maciej Makowski (Aurora, CA); Eli Joshua Fidler (Toronto, CA); Sasitheran Shanmugarajah (Pickering, CA)

Assignee: Amazon Technologies, Inc.

G10L15/22G10L15/02G10L15/30G10L2015/223

Abstract

Claims (104)

1. A computer-implemented method, the method comprising:

determining first audio data by a first device that includes at least one microphone and is configured to perform speech processing, the first audio data representing a first utterance captured by the first device;

beginning speech processing on the first audio data by the first device;

after beginning the speech processing on the first audio data, receiving, by the first device and from a second device, second audio data representing a second utterance;

determining a first number of utterances that the first device is configured to process at a time;

determining a second number of utterances that the first device is currently processing, the second number including the first utterance represented by the first audio data;

determining that the first number is equal to the second number;

in response to determining that the first number is equal to the second number, sending the second audio data to a natural language system for processing;

determining a first action responsive to the first utterance based on the speech processing of the first audio data;

causing the first action to be performed;

receiving, from the natural language system, a command to perform a second action responsive to the second utterance; and

causing the second action to be performed.

2. The computer-implemented method of claim 1 , further comprising:

receiving, by the first device, third audio data representing a third utterance;

beginning speech processing on the third audio data by the first device;

after beginning the speech processing on the third audio data, receiving, by the first device, fourth audio data representing a fourth utterance;

determining that the first device is currently processing the third audio data;

determining a plurality of devices connected to a wireless network associated with the first device;

identifying, from the plurality of devices, a third device configured to perform speech processing; and

sending the fourth audio data to the third device for processing.

3. The computer-implemented method of claim 1 , further comprising:

receiving, by the first device, third audio data representing a third utterance;

beginning speech processing on the third audio data by the first device;

after beginning the speech processing on the third audio data, receiving, by the first device and from a third device, fourth audio data representing a fourth utterance;

determining that the first device is currently processing the third audio data;

identifying a fourth device configured to perform speech processing;

sending the fourth audio data to the fourth device for processing; and

sending, by the first device to the fourth device, a command instructing the fourth device to communicate with the third device.

4. The computer-implemented method of claim 1 , further comprising:

receiving, by the first device, third audio data representing a third utterance;

beginning speech processing on the third audio data by the first device;

after beginning the speech processing on the third audio data, receiving, by the first device and from a third device, fourth audio data representing a fourth utterance;

determining that the first device is currently processing the third audio data;

in response to determining that the first device is currently processing the third audio data, identifying a fourth device configured to perform speech processing;

generating a command instructing the third device to send the fourth audio data to the fourth device; and

sending the command to the third device.

5. A computer-implemented method, the method comprising:

determining, by a first device including at least one microphone and configured to perform speech processing, first audio data representing a first utterance captured by the first device;

beginning speech processing on the first audio data by the first device;

receiving, by the first device while performing speech processing on the first audio data, second audio data from a second device, the second audio data representing a second utterance;

determining that the first device is currently processing the first audio data;

in response to determining that the first device is currently processing the first audio data, performing at least one operation that results in the second audio data being processed by a third device;

determining a first action responsive to the first utterance based on the speech processing of the first audio data; and

causing the first action to be performed.

6. The computer-implemented method of claim 5 , wherein the third device corresponds to a natural language processing system, the method further comprising:

receiving, from the third device, a command to perform a second action responsive to the second utterance; and

causing the second action to be performed.

7. The computer-implemented method of claim 5 , wherein performing the at least one operation further comprises:

determining a plurality of devices connected to a wireless network associated with the first device;

identifying, from the plurality of devices, the third device, the third device configured to perform speech processing; and

sending the second audio data to the third device for processing.

8. The computer-implemented method of claim 5 , wherein determining that the first device is currently processing the first audio data further comprises:

determining a first number of utterances that the first device is configured to process at a time;

determining a second number of utterances that the first device is currently processing, the second number including the first utterance represented by the first audio data; and

determining that the first number is equal to the second number.

9. The computer-implemented method of claim 5 , further comprising, prior to beginning speech processing of the first audio data:

receiving, by the first device from a fourth device, third audio data representing the first utterance;

determining that the first audio data was captured before the third audio data;

in response to determining that the first audio data was captured before the third audio data, determining to process the first audio data; and

sending, to the fourth device, a command to stop capturing the third audio data.

10. The computer-implemented method of claim 5 , further comprising:

receiving, by the first device after beginning speech processing on the first audio data, a request to perform a second action; and

prior to completing the speech processing on the first audio data, causing the second action to be performed.

11. The computer-implemented method of claim 5 , wherein performing the at least one operation further comprises:

sending the second audio data to the third device for processing; and

sending, by the first device to the third device, an indication for the third device to communicate with the second device.

12. The computer-implemented method of claim 5 , wherein performing the at least one operation further comprises:

generating an indication corresponding to the third device performing speech processing on the third audio data; and

sending the indication to the second device.

13. A system comprising:

at least one processor; and

memory including instructions operable to be executed by the at least one processor to cause the system to:

determine, by a first device including at least one microphone and configured to perform speech processing, first audio data representing a first utterance captured by the first device;

begin speech processing on the first audio data by the first device;

receive, by the first device while performing speech processing on the first audio data, second audio data from a second device, the second audio data representing a second utterance;

determine that the first device is currently processing the first audio data;

in response to determining that the first device is currently processing the first audio data, perform at least one operation that results in the second audio data being processed by a third device;

determine a first action responsive to the first utterance based on the speech processing of the first audio data; and

cause the first action to be performed.

14. The system of claim 13 , wherein the third device corresponds to a natural language processing system, and the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

receive, from the third device, a command to perform a second action responsive to the second utterance; and

cause the second action to be performed.

15. The system of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to, prior to sending the second audio data to the third device:

determine a plurality of devices connected to a wireless network associated with the first device;

identify, from the plurality of devices, the third device, the third device configured to perform speech processing; and

send the second audio data to the third device for processing.

16. The system of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine a first number of utterances that the first device is configured to process at a time;

determine a second number of utterances that the first device is currently processing, the second number including the first utterance represented by the first audio data; and

determine that the first number is equal to the second number.

17. The system of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to, prior to beginning speech processing of the first audio data:

receive, by the first device from a fourth device, third audio data representing the first utterance;

determine that the first audio data was captured before the third audio data;

in response to determining that the first audio data was captured before the third audio data, determine to process the first audio data; and

send, to the fourth device, a command to stop capturing the third audio data.

18. The system of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

receive, by the first device after beginning speech processing on the first audio data, a request to perform a second action; and

prior to completing the speech processing on the first audio data, cause the second action to be performed.

19. The system of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

send the second audio data to the third device for processing; and

send, by the first device to the third device, an indication for the third device to communicate with the second device.

20. The system of claim 13 , wherein the memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

generate an indication corresponding to the third device performing speech processing on the third audio data; and

send the indication to the second device.

Assignments (1)

Cited By (11)

US 12,198,689 US 12,236,955 US 12,332,533 US 12,360,737 US 12,400,651 US 12,444,418 US 12,562,173 US 12,597,425 US 12,626,707 US 12,664,984 US 12,739,293