IP Library › Granted Patent US 11,568,867
Granted Patent B2
US 11,568,867 · App. 16/933,904 · Granted Jan 31, 2023

Detecting self-generated wake expressions

Inventors: Michael Alan Pogue (Sunnyvale, CA); Philip Ryan Hilmes (Sunnyvale, CA)
Assignee: Amazon Technologies, Inc.
G10L15/22G10L2015/088G10L2021/02087G10L2021/02166
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,568,867
App. No.
16/933,904
Granted
Jan 31, 2023
Kind
B2
Abstract

A speech-based audio device may be configured to detect a user-uttered wake expression. For example, the audio device may generate a parameter indicating whether output audio is currently being produced by an audio speaker, whether the output audio contains speech, whether the output audio contains a predefined expression, loudness of the output audio, loudness of input audio, and/or an echo characteristic. Based on the parameter, the audio device may determine whether an occurrence of the predefined expression in the input audio is a result of an utterance of the predefined expression by a user.

Claims (64)

1. A device comprising:

a housing;

a first microphone disposed proximate a top of the housing;

a second microphone disposed proximate the top of the housing;

a speaker disposed proximate a bottom of the housing and directed at least partly away from the first microphone and the second microphone;

one or more processors; and

one or more non-transitory storage media storing computer-executable instructions that, when executed by the one or more processors, cause the system to:

generate, at a first time and using the first microphone, first audio data corresponding to sound;

generate, at a second time and using the second microphone, second audio data corresponding to the sound;

determine a difference between the first time and the second time;

generate, based at least in part on the difference, first beamforming data using the first audio data and the second audio data, the first beamforming data corresponding to a first direction;

generate, based at least in part on the difference, second beamforming data using the first audio data and the second audio data, the second beamforming data corresponding to a second direction;

perform speech recognition on the first beamforming data;

determine that the first beamforming data includes a predefined expression;

perform speech recognition on the second beamforming data;

determine that the second beamforming data does not include the predefined expression;

determine, based at least in part on determining that the first beamforming data includes the predefined expression and the second beamforming data does not include the predefined expression, that the sound originated from a user; and

send subsequent audio data to a remote computing device.

2. The device of claim 1 , wherein generating the first beamforming data comprises generating directional audio data that emphasizes a first portion of at least one of the first audio data or the second audio data with respect to a second portion of at least one of the first audio data or the second audio data.

3. The device of claim 1 , wherein the first microphone and the second microphone are directed upward from a top portion of the device.

4. The device of claim 1 , wherein the first microphone and the second microphone comprise at least a portion of a circular arrangement of microphones.

5. The device of claim 1 , wherein generating the first beamforming data comprises:

determining a first phase associated with the first audio data or the second audio data; and

determining a second phase associated with the first audio data or the second audio data, the first beamforming data being based at least partly on the first phase and the second phase.

6. The device of claim 1 , wherein the sound comprises a first sound and the computer-executable instructions, when executed by the one or more processors, further cause the device to:

generate reference data based at least in part on second sound, the reference data corresponding to a pattern of input signals.

7. The device of claim 1 , wherein the computer-executable instructions, when executed by the one or more processors, further cause the device to:

generate, based at least in part on the predefined expression, an audio output by an audio speaker of the device.

8. A device comprising:

a housing;

a plurality of microphones disposed proximate a top of the housing;

a speaker disposed proximate a bottom of the housing and directed at least partly away from the plurality of microphones;

one or more processors; and

one or more non-transitory storage media storing computer-executable instructions that, when executed by the one or more processors, cause the device to:

generate, at a first time and using a first microphone of the plurality of microphones, first audio data corresponding to sound;

generate, at a second time and using a second microphone of the plurality of microphones, second audio data corresponding to the sound;

determine a difference between the first time and the second time;

generate, based at least in part on the difference, first beamforming data by increasing an amplitude of a portion of at least one of the first audio data or the second audio data, the first beamforming data corresponding to a first direction;

generate, based at least in part on the difference, second beamforming data by increasing an amplitude of the portion of at least one of the first audio data or the second audio data, the second beamforming data corresponding to a second direction;

determine that the first beamforming data includes a predefined expression;

determine that the second beamforming data does not include the predefined expression;

determine, based at least in part on determining that the first beamforming data includes the predefined expression and the second beamforming data does not include the predefined expression, that the sound originated from a user; and

send subsequent audio data to a remote computing device.

9. The device of claim 8 , wherein the plurality of microphones are positioned at an upper portion of the device and further comprising an audio speaker positioned at a lower portion of the device.

10. The device of claim 8 , wherein the plurality of microphones are positioned at the device evenly spaced apart or linearly.

11. The device of claim 8 , wherein the plurality of microphones comprise a circular arrangement at an upper portion of the device.

12. The device of claim 11 , wherein the housing comprises a cylindrically-shaped housing to house one or more components of the device.

13. The device of claim 12 , wherein the plurality of microphones are positioned at a top surface of the cylindrically-shaped housing of the device.

14. The device of claim 8 , wherein the computer-executable instructions, when executed by the one or more processors, further cause the device to:

generate an audio output by an audio speaker of the device.

15. A method implemented at least in part by a device, the method comprising:

generating, at a first time and using a first microphone disposed proximate a top of a housing of the device, first audio data corresponding to sound;

generating, at a second time and using a second microphone disposed proximate the top of the housing, second audio data corresponding to the sound;

determining a difference between the first time and the second time;

generating, based at least in part on the difference, first beamforming audio data using the first audio data and the second audio data, the first beamforming audio data corresponding to a first direction;

generating, based at least in part on the difference, second beamforming audio data using the first audio data and the second audio data, the second beamforming audio data corresponding to a second direction;

performing speech recognition on the first beamforming audio data;

determining that the first beamforming audio data includes a trigger expression;

performing speech recognition on the second beamforming audio data;

determining, based at least in part on determining that the first beamforming audio data includes the trigger expression and the second beamforming audio data does not include the trigger expression, that the sound originated from a user;

sending subsequent audio data to a remote computing device; and

outputting third audio data using a speaker disposed proximate a bottom of the housing and directed at least partly away from the first microphone and the second microphone.

16. The method of claim 15 , wherein the first beamforming audio data comprises a portion of at least one of the first audio data or the second audio data having an increased amplitude.

17. The method of claim 15 , wherein the sound comprises first sound and further comprising generating reference data based at least partly on second sound, the reference data corresponding to a pattern of input signals.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2020
From: POGUE, MICHAEL ALAN; HILMES, PHILIP RYAN
To: RAWLES LLC
Reel/Frame 053258/0121 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2020
From: RAWLES LLC
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 053258/0157 →
Continuity (3)
Continuation 15652019 · Jul 17, 2017
Continuation 13929540 · Jun 27, 2013
Related Publication 20210005197A1 · Jan 7, 2021
Cited By (1)
US 12,516,463