IP Library Granted Patent US 12,249,331
Granted Patent B2
US 12,249,331 · App. 18/144,465 · Granted Mar 11, 2025

Multi-layer keyword detection

Inventors: Christopher Wayne Lockhart (Louisville, KY); Matthew Joseph Cole (Arlington, MA); Xulei Liu (Brighton, MA)
Assignee: Amazon Technologies, Inc.
G10L15/22G10L15/02G10L15/08G10L15/26G10L2015/025G10L2015/088G10L2015/223G10L15/30G10L15/32
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,249,331
App. No.
18/144,465
Granted
Mar 11, 2025
Kind
B2
Abstract

A system and method for temporarily disabling keyword detection to avoid detection of machine-generated keywords. A local device may operate two keyword detectors. The first keyword detector operates on input audio data received by a microphone to capture keywords uttered by a user. In these instances, the keyword may be detected by the first detector and the audio data may be indicated for speech processing. The system may determine output audio data responsive to the input audio data. The local device may process the output audio data to determine that it also includes the keyword. The device may then disable the first keyword detector while the output audio data is played back by an audio speaker of the local device. Thus the local device may avoid detection of a keyword originating from the output audio. The first keyword detector may be reactivated after a time interval during which the keyword might be detectable in the output audio.

Claims (36)

1. A computer-implemented method, comprising:

determining first audio data corresponding to audio to be output;

processing the first audio data to detect a keyword;

after determining the first audio data includes the keyword, entering a second operation mode of a first device to disregard the keyword; and

after entering the second operation mode of the first device, causing the audio to be output.

2. The computer-implemented method of claim 1 , wherein processing the first audio data to detect the keyword is performed by a second device different from the first device.

3. The computer-implemented method of claim 1 , wherein causing the audio to be output comprises causing the audio to be output using a second device different from the first device.

4. The computer-implemented method of claim 1 , wherein:

processing the first audio data to detect the keyword is performed by a second device different from the first device; and

causing the audio to be output comprises causing the audio to be output using a third device different from the first device and different from the second device.

5. The computer-implemented method of claim 1 , wherein processing the first audio data to detect the keyword is performed by the first device.

6. The computer-implemented method of claim 1 , wherein:

processing the first audio data to detect the keyword comprises using a first keyword detection component; and

entering the second operation mode adjusting operation of the first device comprises adjusting operation of a second keyword detection component different from the first keyword detection component.

7. The computer-implemented method of claim 6 , wherein processing the first audio data to detect the keyword comprises using the first keyword detection component of the first device.

8. The computer-implemented method of claim 1 , wherein causing the audio to be output comprises causing the audio to be output using the first device.

9. The computer-implemented method of claim 1 , wherein the keyword comprises a wakeword.

10. A system comprising:

at least one processor; and

at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:

determine first audio data corresponding to audio to be output;

process the first audio data to detect a keyword;

after determination that the first audio data includes the keyword, enter a second operation mode of a first device to disregard the keyword; and

after entering the second operation mode of the first device, cause the audio to be output.

11. The system of claim 10 , wherein processing of the first audio data to detect the keyword is performed by a second device different from the first device.

12. The system of claim 10 , wherein the instructions that cause the system to cause the audio to be output comprise instructions that, when executed by the at least one processor, cause the system to cause the audio to be output using a second device different from the first device.

13. The system of claim 10 , wherein:

processing of the first audio data to detect the keyword is performed by a second device different from the first device; and

the instructions that cause the system to cause the audio to be output comprise instructions that, when executed by the at least one processor, cause the system to cause the audio to be output using a third device different from the first device and different from the second device.

14. The system of claim 10 , wherein processing of the first audio data to detect the keyword is performed by the first device.

15. The system of claim 10 , wherein:

the instructions that cause the system to process the first audio data to detect the keyword comprise instructions that, when executed by the at least one processor, cause the system to use a first keyword detection component; and

the instructions that cause the system to enter the second operation mode of the first device comprise instructions that, when executed by the at least one processor, cause the system to adjust operation of a second keyword detection component different from the first keyword detection component.

16. The system of claim 15 , wherein the instructions that cause the system to process the first audio data to detect the keyword comprise instructions that, when executed by the at least one processor, cause the system to use the first keyword detection component of the first device.

17. The system of claim 10 , wherein the instructions that cause the system to cause the audio to be output comprise instructions that, when executed by the at least one processor, cause the system to cause the audio to be output using the first device.

18. The system of claim 10 , wherein the keyword comprises a wakeword.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 8, 2023
From: LOCKHART, CHRISTOPHER WAYNE; COLE, MATTHEW JOSEPH; LIU, XULEI
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 063566/0239 →
Continuity (5)
Continuation 17590406 · Feb 1, 2022
Continuation 16783826 · Feb 6, 2020
Continuation 16131705 · Sep 14, 2018
Continuation 15370216 · Dec 6, 2016
Related Publication 20230360647A1 · Nov 9, 2023
References Cited (100)
US 4658426A · Chabries et al. · 1987 [cited by applicant]
US 5199080A · Kimura et al. · 1993 [cited by applicant]
US 5708704A · Fisher · 1998 [cited by applicant]
US 5832440A · Woodbridge et al. · 1998 [cited by applicant]
US 6094476A · Hunt et al. · 2000 [cited by applicant]
US 6098043A · Forest et al. · 2000 [cited by applicant]
US 6138100A · Dutton et al. · 2000 [cited by applicant]
US 6603835B2 · Engelke et al. · 2003 [cited by applicant]
US 6591239B1 · McCall et al. · 2003 [cited by applicant]
US 6606595B1 · Chengalvarayan et al. · 2003 [cited by applicant]
US 6650997B2 · Funk · 2003 [cited by applicant]
US 6725193B1 · Makovicka · 2004 [cited by applicant]
US 6801889B2 · Walker · 2004 [cited by applicant]
US 6845356B1 · Epstein · 2005 [cited by applicant]
US 7117442B1 · Kemble et al. · 2006 [cited by applicant]
US 7318030B2 · Guduru · 2008 [cited by applicant]
US 7392188B2 · Junkawitsch et al. · 2008 [cited by applicant]
US 7437286B2 · Pi et al. · 2008 [cited by applicant]
US 7519122B2 · Hayashi · 2009 [cited by applicant]
US 7524131B2 · Schwarz et al. · 2009 [cited by applicant]
US 7624016B2 · Ju et al. · 2009 [cited by applicant]
US 7747446B2 · Blass et al. · 2010 [cited by applicant]
US 7848314B2 · Cross et al. · 2010 [cited by applicant]
US 8117036B2 · Bedingfield · 2012 [cited by applicant]
US 8359202B2 · Kurzweil et al. · 2013 [cited by applicant]
US 8442833B2 · Chen · 2013 [cited by applicant]
US 8666732B2 · Sato et al. · 2014 [cited by applicant]
US 8738382B1 · Herz · 2014 [cited by applicant]
US 8768712B1 · Sharifi · 2014 [cited by applicant]
US 8775191B1 · Sharifi · 2014 [cited by examiner]
US 8928579B2 · Wilson et al. · 2015 [cited by applicant]
US 8972262B1 · Buryak · 2015 [cited by applicant]
US 9014358B2 · Garcia · 2015 [cited by applicant]
US 9047857B1 · Barton · 2015 [cited by applicant]
US 9258302B2 · Fu et al. · 2016 [cited by applicant]
US 9318107B1 · Sharifi · 2016 [cited by applicant]
US 9318129B2 · Vasilieff et al. · 2016 [cited by applicant]
US 9343073B1 · Murgia et al. · 2016 [cited by applicant]
US 9497544B2 · Mohammad et al. · 2016 [cited by applicant]
US 9576591B2 · Kim · 2017 [cited by applicant]
US 9584642B2 · Kadiwala et al. · 2017 [cited by applicant]
US 9620144B2 · Parkinson et al. · 2017 [cited by applicant]
US 9654437B2 · Shen et al. · 2017 [cited by applicant]
US 9691378B1 · Meyers et al. · 2017 [cited by applicant]
US 9704486B2 · Basye et al. · 2017 [cited by applicant]
US 9747899B2 · Pogue et al. · 2017 [cited by applicant]
US 9774747B2 · Garland et al. · 2017 [cited by applicant]
US 9779724B2 · Gruenstein et al. · 2017 [cited by applicant]
US 9886953B2 · Lemay et al. · 2018 [cited by applicant]
US 9984675B2 · Schuster et al. · 2018 [cited by applicant]
US 10037763B2 · Riedmiller et al. · 2018 [cited by applicant]
US 10044854B2 · Rae et al. · 2018 [cited by applicant]
US 10062384B1 · Sugiura et al. · 2018 [cited by applicant]
US 10102848B2 · Koulomzin · 2018 [cited by applicant]
US 10102853B2 · Page · 2018 [cited by examiner]
US 10147444B2 · Wang et al. · 2018 [cited by applicant]
US 10152966B1 · O'Malley et al. · 2018 [cited by applicant]
US 10192557B2 · Lee et al. · 2019 [cited by applicant]
US 10199040B2 · Lahiri et al. · 2019 [cited by applicant]
US 10475445B1 · Meyers et al. · 2019 [cited by applicant]
US 10475449B2 · Lang · 2019 [cited by applicant]
US 10490192B2 · Ogawa · 2019 [cited by applicant]
US 10747498B2 · Stasior et al. · 2020 [cited by applicant]
US 10748531B2 · Kim · 2020 [cited by applicant]
US 10930266B2 · Meyers et al. · 2021 [cited by applicant]
US 20020042709A1 · Klisch et al. · 2002 [cited by applicant]
US 20020055844A1 · L'Esperance et al. · 2002 [cited by applicant]
US 20030138118A1 · Stahl · 2003 [cited by applicant]
US 20030217161A1 · Balasuriya · 2003 [cited by examiner]
US 20040204933A1 · Walker · 2004 [cited by applicant]
US 20050203749A1 · Miyata · 2005 [cited by applicant]
US 20060085199A1 · Jain · 2006 [cited by applicant]
US 20060149546A1 · Runge et al. · 2006 [cited by applicant]
US 20080015845A1 · Schmidt et al. · 2008 [cited by applicant]
US 20080091415A1 · Schafer · 2008 [cited by applicant]
US 20080140412A1 · Millman et al. · 2008 [cited by applicant]
US 20090086934A1 · Thomas · 2009 [cited by applicant]
US 20100081487A1 · Chen et al. · 2010 [cited by applicant]
US 20110238417A1 · Yamamoto et al. · 2011 [cited by applicant]
US 20130097510A1 · Maling, III · 2013 [cited by examiner]
US 20130238326A1 · Kim et al. · 2013 [cited by applicant]
US 20150006176A1 · Pogue · 2015 [cited by examiner]
US 20150116211A1 · Kim · 2015 [cited by examiner]
US 20150161989A1 · Hsu · 2015 [cited by examiner]
US 20150245154A1 · Dadu · 2015 [cited by examiner]
US 20150302855A1 · Kim · 2015 [cited by examiner]
US 20160055847A1 · Dahan · 2016 [cited by applicant]
US 20160086603A1 · Rosner · 2016 [cited by examiner]
US 20160171975A1 · Sun · 2016 [cited by examiner]
US 20160171976A1 · Sun · 2016 [cited by examiner]
US 20160189726A1 · Raniwala et al. · 2016 [cited by applicant]
US 20160267913A1 · Kim · 2016 [cited by examiner]
US 20170100036A1 · Cinbis · 2017 [cited by examiner]
US 20170229119A1 · Jang · 2017 [cited by examiner]
US 20170345422A1 · Yang · 2017 [cited by examiner]
US 20180005633A1 · Bocklet · 2018 [cited by examiner]
US 20180108343A1 · Stevans · 2018 [cited by examiner]
US 20190295555A1 · Wilberding · 2019 [cited by examiner]
US 20210210071A1 · Meyers et al. · 2021 [cited by applicant]
CN 102668391A · 2012 [cited by examiner]