IP Library Granted Patent US 11,322,152
Granted Patent B2
US 11,322,152 · App. 16/443,160 · Granted May 3, 2022

Speech recognition power management

Inventors: Kenneth John Basye (Sutton, MA); Hugh Evan Secker-Walker (Newburyport, MA); Tony David (San Jose, CA); Reinhard Kneser (North Rhine-Westphalia, DE); Jeffrey Penrod Adams (Tyngsborough, MA); Stan Weidner Salvador (Tega Cay, SC); Mahesh Krishnamoorthy (Melrose, MA)
Assignee: Amazon Technologies, Inc.
G10L15/28G10L15/30G10L25/78G10L2015/088
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,322,152
App. No.
16/443,160
Granted
May 3, 2022
Kind
B2
Abstract

Power consumption for a computing device may be managed by one or more keywords. For example, if an audio input obtained by the computing device includes a keyword, a network interface module and/or an application processing module of the computing device may be activated. The audio input may then be transmitted via the network interface module to a remote computing device, such as a speech recognition server. Alternately, the computing device may be provided with a speech recognition engine configured to process the audio input for on-device speech recognition.

Claims (49)

1. A system comprising:

an audio input component comprising a microphone, wherein the audio input component is configured to generate audio data representing sound detected by the microphone;

a first set of one or more processors, configured to:

determine that the audio data likely comprises data representing voice activity; and

determine, in response to determining that the audio data likely comprises data representing the voice activity, that the audio data likely comprises data representing a designated keyword; and

a second set of one or more processors, configured to:

determine that the audio data likely does not comprise data representing the designated keyword; and

generate, in response to determining that the audio data likely does not comprise data representing the designated keyword, an instruction that prevents transmission of the audio data.

2. The system of claim 1 , wherein the designated keyword comprises a wakeword indicative of device-directed speech.

3. The system of claim 1 , wherein the first set of one or more processors configured to determine that the audio data likely comprises data representing the voice activity is further configured to determine that an energy level represented by the audio data is equal to or greater than an energy level threshold.

4. The system of claim 1 , wherein the first set of one or more processors configured to determine that the audio data likely comprises data representing the voice activity is further configured to determine at least one of a spectral slope between two frames of the audio data or a signal-to-noise ratio of the audio data within a spectral band.

5. The system of claim 1 , wherein the first set of one or more processors configured to determine that the audio data likely comprises data representing the voice activity comprises a first digital signal processor, and wherein the first digital signal processor is further configured to activate a second digital processor in response to determining that the audio data likely comprises data representing the voice activity.

6. The system of claim 1 , wherein the first set of one or more processors configured to determine that the audio data likely comprises data representing a designated keyword comprises a digital signal processor, and wherein the digital signal processor is further configured to activate a microprocessor in response to determining that the audio data likely comprises data representing the designated keyword.

7. The system of claim 1 , further comprising a network interface component,

wherein the audio input component is further configured to generate second audio data representing sound detected by the microphone;

wherein the first set of one or more processors is further configured to:

determine that the second audio data likely comprises data representing voice activity; and

determine, in response to determining that the second audio data likely comprises data representing second voice activity, that the second audio data likely comprises data representing the designated keyword; and

wherein the second set of one or more processors is further configured to:

determine that the second audio data likely comprises data representing the designated keyword;

cause the network interface component to send a transmission of at least a portion of the second audio data to a remote computing system, wherein the portion of the second audio data represents an utterance; and

receive speech recognition results from the remote computing system.

8. The system of claim 7 , further comprising a speaker, wherein the speech recognition results comprise third audio data representing an audio output, and wherein the second set of one or more processors is further configured to cause the speaker to present the audio output.

9. The system of claim 7 , wherein the speech recognition results comprise text data representing the utterance, and wherein the second set of one or more processors is further configured to determine an audio response to the utterance using the speech recognition results.

10. The system of claim 1 , wherein the second set of one or more processors is further configured to generate speech recognition results for an utterance represented by the audio data.

11. The system of claim 1 , wherein the second set of one or more processors are deactivated in response to the second set of one or more processors determining that the audio data likely does not comprise data representing the designated keyword.

12. A computer-implemented method comprising:

under control of a computing system comprising a plurality of processors,

receiving audio data representing sound detected by a microphone;

determining, by a first subset of the plurality of processors, that the audio data likely comprises data representing voice activity based at least partly on one of: a difference between two or more frames of the audio data; a classification model; or a state model;

in response to determining that the audio data likely comprises data representing the voice activity, determining, by the first subset of the plurality processors, that the audio data likely comprises data representing a designated keyword; and

in response to determining that the audio data likely comprises data representing the designated keyword:

performing, by a second subset of the plurality of processors, speech recognition on at least a portion of the audio data to obtain speech recognition results;

determining, by the second subset of the plurality of processors, that the audio data likely does not comprise data representing the designated keyword; and

generating, by the second subset of the plurality of processors in response to determining that the audio data likely does not comprise data representing the designated keyword, an instruction that prevents transmission of the audio data.

13. The computer-implemented method of claim 12 , further comprising activating, by the first subset of the plurality of processors, a digital signal processor in response to determining that the audio data likely comprises data representing the voice activity, wherein the first subset of the plurality of processors comprises the digital signal processor, and wherein the determining that the audio data likely comprises data representing the designated keyword is performed using the digital signal processor.

14. The computer-implemented method of claim 12 , further comprising:

receiving second audio data representing sound detected by the microphone;

determining, by the first subset of the plurality of processors, that the second audio data likely comprises data representing voice activity;

in response to determining that the second audio data likely comprises data representing voice activity, determining, by the first subset of the plurality of processors, that the second audio data likely comprises data representing the designated keyword;

determining, by the second subset of the plurality of processors, that the second audio data likely comprises data representing the designated keyword;

sending at least a portion of the second audio data to a remote computing system; and

receiving second speech recognition results from the remote computing system.

15. The computer-implemented method of claim 14 , further comprising presenting audio output using the second speech recognition results, wherein the second speech recognition results comprise third audio data representing the audio output.

16. The computer-implemented method of claim 12 , further comprising:

activating at least one processor of the second subset of the plurality of processors in response to the determining, by the first subset of the plurality of processors, that the second audio data likely comprises data representing the designated keyword; and

deactivating the at least one processor of the second subset of the plurality of processors in response to the determining, by the second subset of the plurality of processors, that the audio data likely does not comprise data representing the designated keyword.

17. The computer-implemented method of claim 12 , wherein the determining, by the first subset of the plurality of processors, that the audio data likely comprises data representing voice activity comprises determining, by the first subset of the plurality of processors, that the audio data likely comprises data representing voice activity based on a spectral slope between the two or more frames of the audio data.

18. The computer-implemented method of claim 12 , wherein the determining, by the first subset of the plurality of processors, that the audio data likely comprises data representing voice activity comprises determining, by the first subset of the plurality of processors, that the audio data likely comprises data representing voice activity based on one of: a linear classifier, a support vector machine, a decision tree, a hidden Markov model, or Gaussian mixture model.

Continuity (3)
Continuation 15645918 · Jul 10, 2017
Continuation 13711510 · Dec 11, 2012
Related Publication 20200043499A1 · Feb 6, 2020