IP Library › Granted Patent US 9,934,783
Granted Patent B2
US 9,934,783 · App. 15/176,482 · Granted Apr 3, 2018

Hotword recognition

Inventors: Matthew Sharifi (Kilchberg, CH); Jakob Nicolaus Foerster (Oxford, GB)
Assignee: Google LLC
G10L15/22G06F21/31G10L15/08G10L17/08G10L2015/088G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,934,783
App. No.
15/176,482
Filed
Jun 8, 2016
Granted
Apr 3, 2018
Kind
B2
Art Unit
2659
USPC
704/239
Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving audio data corresponding to an utterance, determining that the audio data corresponds to a hotword, generating a hotword audio fingerprint of the audio data that is determined to correspond to the hotword, comparing the hotword audio fingerprint to one or more stored audio fingerprints of audio data that was previously determined to correspond to the hotword, detecting whether the hotword audio fingerprint matches a stored audio fingerprint of audio data that was previously determined to correspond to the hotword based on whether the comparison indicates a similarity between the hotword audio fingerprint and one of the one or more stored audio fingerprints that satisfies a predetermined threshold, and in response to detecting that the hotword audio fingerprint matches a stored audio fingerprint, disabling access to a computing device into which the utterance was spoken.

Claims (37)

1. A computer-implemented method comprising:

receiving, by a hotword detector of a mobile computing device that includes (a) the hotword detector, and (b) a replay attack engine and that is (i) operating in a mode in which access to one or more resources is disabled, and (ii) is configured to exit the mode in which access to the one or more resources is disabled based on receiving audio input corresponding to an utterance of a hotword, an audio input corresponding to a recording of an utterance of a hotword that was previously input to the same mobile computing device; and

in response to receiving, by the mobile computing device, the audio input corresponding to a recording of an utterance of a hotword that was previously input to the same mobile computing device, preventing, by the replay attack engine of the mobile computing device, the mobile computing device from exiting the mode in which access to the one or more resources is disabled.

2. The computer-implemented method of claim 1 , wherein the utterance of the hotword that was previously input is stored in a database.

3. The computer-implemented method of claim 1 , wherein the hotword is a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.

4. The computer-implemented method of claim 1 , further comprising:

determining that the audio input corresponds to an utterance of a hotword that was previously input based on a similarity between the audio input and one or more stored utterances of the hotword.

5. The computer-implemented method of claim 1 , wherein preventing the mobile computing device from exiting the mode in which access to the one or more resources is disabled comprises one or more of: preventing the mobile computing device from being unlocked, locking the mobile computing device, initiating an authentication process, and preventing the mobile computing device from waking.

6. The computer-implemented method of claim 1 , further comprising:

receiving, by the mobile computing device, additional audio data corresponding to a voice command or query; and

determining a type of the voice command or query.

7. The computer-implemented method of claim 1 , further comprising:

storing the audio input corresponding to a recording of an utterance of a hotword that was previously input to the same mobile computing device in a database.

8. A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving, by a hotword detector of a mobile computing device that includes (a) the hotword detector, and (b) a replay attack engine and that is (i) operating in a mode in which access to one or more resources is disabled, and (ii) is configured to exit the mode in which access to the one or more resources is disabled based on receiving audio input corresponding to an utterance of a hotword, an audio input corresponding to a recording of an utterance of a hotword that was previously input to the same mobile computing device; and

in response to receiving, by the mobile computing device, the audio input corresponding to a recording of an utterance of a hotword that was previously input to the same mobile computing device, preventing, by the replay attack engine of the mobile computing device, the mobile computing device from exiting the mode in which access to the one or more resources is disabled.

9. The system of claim 8 , wherein the utterance of the hotword that was previously input is stored in a database.

10. The system of claim 8 , wherein the hotword is a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.

11. The system of claim 8 , further comprising:

determining that the audio input corresponds to an utterance of a hotword that was previously input based on a similarity between the audio input and one or more stored utterances of the hotword.

12. The system of claim 8 , wherein preventing the mobile computing device from exiting the mode in which access to the one or more resources is disabled comprises one or more of: preventing the mobile computing device from being unlocked, locking the mobile computing device, initiating an authentication process, and preventing the mobile computing device from waking.

13. The system of claim 8 , further comprising:

receiving, by the mobile computing device, additional audio data corresponding to a voice command or query; and

determining a type of the voice command or query.

14. The system of claim 8 , further comprising:

storing the audio input corresponding to a recording of an utterance of a hotword that was previously input to the same mobile computing device in a database.

15. A computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, by a hotword detector of a mobile computing device that includes (a) the hotword detector, and (b) a replay attack engine and that is (i) operating in a mode in which access to one or more resources is disabled, and (ii) is configured to exit the mode in which access to the one or more resources is disabled based on receiving audio input corresponding to an utterance of a hotword, an audio input corresponding to a recording of an utterance of a hotword that was previously input to the same mobile computing device; and

in response to receiving, by the mobile computing device, the audio input corresponding to a recording of an utterance of a hotword that was previously input to the same mobile computing device, preventing, by the replay attack engine of the mobile computing device, the mobile computing device from exiting the mode in which access to the one or more resources is disabled.

16. The computer-readable storage device of claim 15 , wherein the utterance of the hotword that was previously input is stored in a database.

17. The computer-readable storage device of claim 15 , wherein the hotword is a particular term that triggers semantic interpretation of an additional term of one or more terms that follow the particular term.

18. The computer-readable storage device of claim 15 , further comprising:

determining that the audio input corresponds to an utterance of a hotword that was previously input based on a similarity between the audio input and one or more stored utterances of the hotword.

19. The computer-readable storage device of claim 15 , wherein preventing the mobile computing device from exiting the mode in which access to the one or more resources is disabled comprises one or more of: preventing the mobile computing device from being unlocked, locking the mobile computing device, initiating an authentication process, and preventing the mobile computing device from waking.

20. The computer-readable storage device of claim 15 , further comprising:

receiving, by the mobile computing device, additional audio data corresponding to a voice command or query; and

determining a type of the voice command or query.

Assignments (2)
CHANGE OF NAME Recorded Oct 5, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044129/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2016
From: SHARIFI, MATTHEW; FOERSTER, JAKOB NICOLAUS
To: GOOGLE INC.
Reel/Frame 038844/0965 →
Continuity (3)
Continuation 14943287 · Nov 17, 2015
Provisional Application 62242650 · Oct 16, 2015
Related Publication 20170110123A1 · Apr 20, 2017