IP Library › Granted Patent US 12,749,485
Granted Patent B2
US 12,749,485 · App. 18/766,909 · Granted Sep 29, 2026

Device leadership negotiation among voice interface devices

Inventors: Kenneth Mixter (Los Altos Hills, CA); Diego Melendo Casado (Mountain View, CA); Alexander H. Gruenstein (Mountain View, CA); Terry Tai (New York, NY); Christopher Thaddeus Hughes (Redwood City, CA); Matthew Nirvan Sharifi (Kilchberg, CH)
Assignee: Google LLC
G10L15/22G10L15/32G10L2015/088G10L2015/223G10L25/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,749,485
App. No.
18/766,909
Granted
Sep 29, 2026
Kind
B2
Abstract

The various implementations described herein include methods and systems for determining device leadership among voice interface devices. In one aspect, a method is performed at a first electronic device of a plurality of electronic devices, each having microphones, a speaker, processors, and memory storing programs for execution by the processors. The first device detects a voice input. It determines a device state and a relevance of the voice input. It identifies a subset of electronic devices from the plurality to which the voice input is relevant. In accordance with a determination that the subset includes the first device, the first device determines a first score of a criterion associated with the voice input and receives second scores of the criterion from other devices in the subset. In accordance with a determination that the first score is higher than the second scores, the first device responds to the detected input.

Claims (36)

1 . A computer-implemented method when executed on data processing hardware of a first voice assistant device causes the data processing hardware to perform operations comprising:

receiving a voice input comprising a hotword and a voice request subsequent to the hotword, the voice input captured by the first voice assistant device and a second voice assistant device, the first voice assistant device and the second voice assistant device each communicatively coupled to a local network implemented at a network interface and configured to respond to voice requests that are subsequent to the hotword;

detecting, in the voice input, the hotword;

based on detecting the hotword, processing the voice input to determine that the voice request comprises a media play request to playback media content on a media output device;

sending, from the first voice assistant device via the local network, a multicast message received by the second voice assistant device, the multicast message received by the second voice assistant device causing the second voice assistant device to not respond to the voice request despite the second voice assistant device capturing the voice input of the hotword and the voice request subsequent to the hotword; and

based on determining that voice request comprises the request to playback the media content on the media output device, causing the media output device to playback the media content.

2 . The computer-implemented method of claim 1 , wherein processing the voice input further comprises processing the voice input to determine that the voice request further comprises a voice designation of the media output device, the voice designation comprising a description of the media output device.

3 . The computer-implemented method of claim 2 , wherein causing the media output device to playback the media content comprises causing the media output device to playback the media content using the voice designation of the media output device.

4 . The computer-implemented method of claim 1 , wherein processing the voice input further comprises processing the voice input to determine that the voice request further comprises a name of a media play application, the media play application streaming the media content from playback on the media output device.

5 . The computer-implemented method of claim 4 , wherein causing the media output device to playback the media content comprises causing the media output device to playback the media content using the media play application.

6 . The computer-implemented method of claim 1 , wherein processing the voice input further comprises processing the voice input to determine that the voice request further comprises a description of the media content.

7 . The computer-implemented method of claim 1 , wherein:

the media output device comprises a speaker; and

the media content played back by the media output device comprises music audibly played back by the media output device.

8 . The computer-implemented method of claim 1 , wherein causing the media output device to playback the media content comprises causing the media output device to playback the media content streamed from a remote content source.

9 . The computer-implemented method of claim 1 , wherein the first voice assistant device and the media output device is communicatively to a local network implemented at a network interface.

10 . The computer-implemented method of claim 1 , wherein the media output device comprises the first voice assistant device.

11 . A first voice activated device comprising:

data processing hardware; and

memory hardware in communication with the data processing hardware and storing instructions that when executed on the data processing hardware causes the data processing hardware to perform operations comprising:

receiving a voice input comprising a hotword and a voice request subsequent to the hotword, the voice input captured by the first voice assistant device and a second voice assistant device, the first voice assistant device and the second voice assistant device each communicatively coupled to a local network implemented at a network interface and configured to respond to voice requests that are subsequent to the hotword;

detecting, in the voice input, the hotword;

based on detecting the hotword, processing the voice input to determine that the voice request comprises a media play request to playback media content on a media output device;

sending, from the first voice assistant device via the local network, a multicast message received by the second voice assistant device, the multicast message received by the second voice assistant device causing the second voice assistant device to not respond to the voice request despite the second voice assistant device capturing the voice input of the hotword and the voice request subsequent to the hotword; and

based on determining that voice request comprises the request to playback the media content on the media output device, causing the media output device to playback the media content.

12 . The first voice assistant device of claim 11 , wherein processing the voice input further comprises processing the voice input to determine that the voice request further comprises a voice designation of the media output device, the voice designation comprising a description of the media output device.

13 . The first voice assistant device of claim 12 , wherein causing the media output device to playback the media content comprises causing the media output device to playback the media content using the voice designation of the media output device.

14 . The first voice assistant device of claim 11 , wherein processing the voice input further comprises processing the voice input to determine that the voice request further comprises a name of a media play application, the media play application streaming the media content from playback on the media output device.

15 . The first voice assistant device of claim 14 , wherein causing the media output device to playback the media content comprises causing the media output device to playback the media content using the media play application.

16 . The first voice assistant device of claim 11 , wherein processing the voice input further comprises processing the voice input to determine that the voice request further comprises a description of the media content.

17 . The first voice assistant device of claim 11 , wherein:

the media output device comprises a speaker; and

the media content played back by the media output device comprises music audibly played back by the media output device.

18 . The first voice assistant device of claim 11 , wherein causing the media output device to playback the media content comprises causing the media output device to playback the media content streamed from a remote content source.

19 . The first voice assistant device of claim 11 , wherein the first voice assistant device and the media output device is communicatively to a local network implemented at a network interface.

20 . The first voice assistant device of claim 11 , wherein the media output device comprises the first voice assistant device.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 9, 2024
From: MIXTER, KENNETH; CASADO, DIEGO MELENDO; GRUENSTEIN, ALEXANDER HOUSTON; TAI, TERRY; HUGHES, CHRISTOPHER THADDEUS; SHARIFI, MATTHEW NIRVAN
To: GOOGLE, INC.
Reel/Frame 067935/0836 →
CHANGE OF NAME Recorded Jul 9, 2024
From: GOOGLE, INC.
To: GOOGLE LLC
Reel/Frame 068255/0395 →
Continuity (10)
Continuation 18312580 · May 4, 2023
Continuation 17242273 · Apr 27, 2021
Continuation 16786943 · Feb 10, 2020
Continuation 16159339 · Oct 12, 2018
Continuation 15788658 · Oct 19, 2017
Continuation 15284483 · Oct 3, 2016
Continuation In Part 15088477 · Apr 1, 2016
Continuation 14675932 · Apr 1, 2015
Provisional Application 62061830 · Oct 9, 2014
Related Publication 20240363113A1 · Oct 31, 2024
References Cited (166)
US 4363102A · Holmgren et al. · 1982 [cited by applicant]
US 5455889A · Bahl · 1995 [cited by examiner]
US 5659665A · Whelpley, Jr. · 1997 [cited by applicant]
US 5838384A · Schindler · 1998 [cited by examiner]
US 5897616A · Kanevsky et al. · 1999 [cited by applicant]
US 6023676A · Erell · 2000 [cited by applicant]
US 6141644A · Kuhn et al. · 2000 [cited by applicant]
US 6567775B1 · Maali et al. · 2003 [cited by applicant]
US 6671672B1 · Heck · 2003 [cited by applicant]
US 6744860B1 · Schrage · 2004 [cited by applicant]
US 6826159B1 · Shaffer et al. · 2004 [cited by applicant]
US 6931375B1 · Bossemeyer, Jr. et al. · 2005 [cited by applicant]
US 6961704B1 · Phillips · 2005 [cited by examiner]
US 6973426B1 · Schier et al. · 2005 [cited by applicant]
US 7016833B2 · Gable et al. · 2006 [cited by applicant]
US 7222072B2 · Chang · 2007 [cited by applicant]
US 7571014B1 · Lambourne et al. · 2009 [cited by applicant]
US 7720012B1 · Borah et al. · 2010 [cited by applicant]
US 7778816B2 · Reynar · 2010 [cited by examiner]
US 7904297B2 · Mirkovic et al. · 2011 [cited by applicant]
US 8099288B2 · Zhang et al. · 2012 [cited by applicant]
US 8200488B2 · Kemp et al. · 2012 [cited by applicant]
US 8209174B2 · Al-Telmissani · 2012 [cited by applicant]
US 8214447B2 · Deslippe et al. · 2012 [cited by applicant]
US 8244534B2 · Qian · 2012 [cited by examiner]
US 8340975B1 · Rosenberger · 2012 [cited by examiner]
US 8588949B2 · Lambourne et al. · 2013 [cited by applicant]
US 8655653B2 · Vos · 2014 [cited by examiner]
US 8670985B2 · Lindahl et al. · 2014 [cited by applicant]
US 8713119B2 · Lindahl · 2014 [cited by applicant]
US 8717949B2 · Crinon et al. · 2014 [cited by applicant]
US 8719009B2 · Baldwin et al. · 2014 [cited by applicant]
US 8719018B2 · Dinerstein · 2014 [cited by applicant]
US 8768687B1 · Quasthoff et al. · 2014 [cited by applicant]
US 8775191B1 · Sharifi et al. · 2014 [cited by applicant]
US 8805890B2 · Zhang et al. · 2014 [cited by applicant]
US 8838457B2 · Cerra et al. · 2014 [cited by applicant]
US 8938394B1 · Faaborg et al. · 2015 [cited by applicant]
US 8996372B1 · Secker-Walker et al. · 2015 [cited by applicant]
US 9142218B2 · Schroeter · 2015 [cited by applicant]
US 9443527B1 · Watanabe · 2016 [cited by examiner]
US 9916839B1 · Scalise · 2018 [cited by examiner]
US 20020049596A1 · Burchard et al. · 2002 [cited by applicant]
US 20020072905A1 · White et al. · 2002 [cited by applicant]
US 20020123890A1 · Kopp et al. · 2002 [cited by applicant]
US 20020193991A1 · Bennett et al. · 2002 [cited by applicant]
US 20030200090A1 · Kawazoe · 2003 [cited by applicant]
US 20030220797A1 · Ito · 2003 [cited by applicant]
US 20030231746A1 · Hunter et al. · 2003 [cited by applicant]
US 20040078191A1 · Tian · 2004 [cited by examiner]
US 20040101112A1 · Kuo · 2004 [cited by applicant]
US 20040158464A1 · Baker · 2004 [cited by examiner]
US 20040236570A1 · Tato · 2004 [cited by examiner]
US 20050165607A1 · Di Fabbrizio et al. · 2005 [cited by applicant]
US 20050171766A1 · Albesano · 2005 [cited by examiner]
US 20060074656A1 · Mathias et al. · 2006 [cited by applicant]
US 20060085188A1 · Goodwin et al. · 2006 [cited by applicant]
US 20060184370A1 · Kwak et al. · 2006 [cited by applicant]
US 20070100620A1 · Tavares · 2007 [cited by applicant]
US 20070198262A1 · Mindlin et al. · 2007 [cited by applicant]
US 20080288256A1 · Agapi · 2008 [cited by examiner]
US 20090240488A1 · White et al. · 2009 [cited by applicant]
US 20090258333A1 · Yu · 2009 [cited by applicant]
US 20090271190A1 · Niemisto · 2009 [cited by examiner]
US 20090292541A1 · Daya et al. · 2009 [cited by applicant]
US 20100070276A1 · Wasserblat et al. · 2010 [cited by applicant]
US 20100110834A1 · Kim et al. · 2010 [cited by applicant]
US 20110026722A1 · Jing et al. · 2011 [cited by applicant]
US 20110054892A1 · Jung et al. · 2011 [cited by applicant]
US 20110060587A1 · Phillips et al. · 2011 [cited by applicant]
US 20110066429A1 · Shperling et al. · 2011 [cited by applicant]
US 20110184730A1 · LeBeau et al. · 2011 [cited by applicant]
US 20110246181A1 · Liang · 2011 [cited by examiner]
US 20110304648A1 · Kim et al. · 2011 [cited by applicant]
US 20120084087A1 · Yang et al. · 2012 [cited by applicant]
US 20120136923A1 · Grube · 2012 [cited by applicant]
US 20120232896A1 · Taleb · 2012 [cited by examiner]
US 20120245941A1 · Cheyer · 2012 [cited by applicant]
US 20120265528A1 · Gruber et al. · 2012 [cited by applicant]
US 20130060571A1 · Soemo et al. · 2013 [cited by applicant]
US 20130124207A1 · Sarin et al. · 2013 [cited by applicant]
US 20130132086A1 · Xu et al. · 2013 [cited by applicant]
US 20130183944A1 · Mozer et al. · 2013 [cited by applicant]
US 20130238326A1 · Kim · 2013 [cited by examiner]
US 20130289996A1 · Fry · 2013 [cited by applicant]
US 20140006825A1 · Shenhav · 2014 [cited by applicant]
US 20140012573A1 · Hung et al. · 2014 [cited by applicant]
US 20140012578A1 · Morioka · 2014 [cited by applicant]
US 20140088961A1 · Woodward et al. · 2014 [cited by applicant]
US 20140156281A1 · Boyd et al. · 2014 [cited by applicant]
US 20140163978A1 · Basye · 2014 [cited by examiner]
US 20140222430A1 · Rao · 2014 [cited by applicant]
US 20140229184A1 · Shires · 2014 [cited by applicant]
US 20140257821A1 · Adams et al. · 2014 [cited by applicant]
US 20140278383A1 · Fan · 2014 [cited by applicant]
US 20140278435A1 · Ganong, III · 2014 [cited by examiner]
US 20150032238A1 · Ondo et al. · 2015 [cited by applicant]
US 20150032443A1 · Karov · 2015 [cited by examiner]
US 20150081295A1 · Yun et al. · 2015 [cited by applicant]
US 20150154953A1 · Bapat et al. · 2015 [cited by applicant]
US 20150228274A1 · Leppanen et al. · 2015 [cited by applicant]
US 20150248885A1 · Koulomzin · 2015 [cited by examiner]
US 20150262577A1 · Nomura · 2015 [cited by applicant]
US 20150379989A1 · Balasubramanian et al. · 2015 [cited by applicant]
US 20160012465A1 · Sharp · 2016 [cited by examiner]
US 20160104483A1 · Foerster et al. · 2016 [cited by applicant]
US 20160155443A1 · Khan et al. · 2016 [cited by applicant]
US 20160189715A1 · Nishikawa · 2016 [cited by applicant]
US 20160217790A1 · Sharifi · 2016 [cited by applicant]
US 20160260431A1 · Newendorp et al. · 2016 [cited by applicant]
US 20160294973A1 · Bakshi et al. · 2016 [cited by applicant]
US 20170083285A1 · Meyers et al. · 2017 [cited by applicant]
US 20180061419A1 · Melendo Casado · 2018 [cited by examiner]
US 20180182390A1 · Hughes · 2018 [cited by examiner]
US 20180196776A1 · Hershko · 2018 [cited by examiner]
US 20190304443A1 · Bhagwan · 2019 [cited by examiner]
JP S59180599A · 1984 [cited by applicant]
JP H0962293A · 1997 [cited by applicant]
JP H09127973A · 1997 [cited by applicant]
JP H1152976A · 1999 [cited by applicant]
JP H11231896A · 1999 [cited by applicant]
JP 2000075954A · 2000 [cited by applicant]
JP 2000310999A · 2000 [cited by applicant]
JP 2003223188A · 2003 [cited by applicant]
JP 2006227634A · 2006 [cited by applicant]
JP 2009104020A · 2009 [cited by applicant]
JP 2011232521A · 2011 [cited by applicant]
JP 201492777A · 2014 [cited by applicant]
JP 2015520409A · 2015 [cited by applicant]
JP 201921428A · 2019 [cited by applicant]
JP 2021184030A · 2021 [cited by applicant]
KR 1020140031391A · 2014 [cited by applicant]
WO 1998040875A1 · 1998 [cited by applicant]
WO 2013006385A1 · 2013 [cited by applicant]
WO 2013163113A1 · 2013 [cited by applicant]
WO 2014008194A1 · 2014 [cited by applicant]
WO 2014130463A2 · 2014 [cited by applicant]
WO 2015025330A1 · 2015 [cited by applicant]
Int'l Search Report and Written Opinion in Int'l Application No. PCT/US2015/052860, mailed Dec. 8, 2015, 12 pages. [cited by applicant]
Japanese Office Action issued in Japanese Application No. 2017-170185, mailed on Oct. 1, 2018, 17 pages (with English Abstract). [cited by applicant]
European Examination Report issued in European Application No. 162007288.8-1210, dated Aug. 1, 2018, 5 pages. [cited by applicant]
Notice of Allowance issued in U.S. Appl. No. 15/278,269, mailed on Jan. 12, 2018, 10 pages. [cited by applicant]
Japanese Office Action issued in Application No. 2016-250670, mailed on Mar. 26, 2018, 7 pages (English Translation). [cited by applicant]
Notice of Allowance in Japanese Application No. 2016-551250, mailed on Aug. 7, 2017, 3 pages (English translation). [cited by applicant]
Office Action issued in U.S. Appl. No. 15/278,269, mailed on Aug. 18, 2017, 23 pages. [cited by applicant]
Office Action issued in Korean Application No. 10-2016-7022701, mailed on Aug. 17, 2017, 4 pages (English translation). [cited by applicant]
Korean Office Action issued in Application No. 10-2016-7027999, mailed on Aug. 28, 2017, 9 pages (with English translation). [cited by applicant]
European Office Action issued in Application No. 15781811.3 mailed on Aug. 23, 2017, 6 pages. [cited by applicant]
Int'l Search Report and Written Opinion issued in Int'l Application No. PCT/US2017/045123, mailed on Nov. 8, 2017, 15 pages. [cited by applicant]
Korean Office Action issued in Application No. 10-2016-7033161, mailed on Dec. 19, 2017, 10 pages (with English translation). [cited by applicant]
Office Action in Japanese Application No. 2016-551250, mailed on Apr. 10, 2017, 6 pages (English translation). [cited by applicant]
Notice of Allowance in Korean Application No. 10-2016-7021778, mailed on Apr. 25, 2017, (English translation) 3 pages. [cited by applicant]
Notice of Allowance in U.S. Appl. No. 15/201,972 mailed on Jun. 1, 2017, 10 pages. [cited by applicant]
Extended European Search Report in European Application No. 16207288.8-1914, mailed on Apr. 4, 2017, 8 pages. [cited by applicant]
Office Action issued in Korean Application No. 10-2016-7027999, mailed Apr. 12, 2017, 9 pages (English translation). [cited by applicant]
Jae-Seung, Choi, “Text-dependent Speaker Recognition using Characteristic Vectors in Speech Signal and Normalized Recognition Method,” Journal of Korean Institute of Information Technology, 10(5), May 2012, 7 pages (Eng… [cited by applicant]
Auckenthaler et al. “Score Normalization for Text-independent Speaker Verification System,” Digital Signal Processing, vol. 10, 2000, 13 pages. [cited by applicant]
Int'l Preliminary Report on Patentability in Int'l Application No. PCT/US2015/052860, mailed on Apr. 11, 2017, 9 pages. [cited by applicant]
Int'l Preliminary Report on Patentability in Int'l Application No. PCT/US2015/030569 mailed on Jan. 24, 2017, 8 pages. [cited by applicant]
Office Action in Korean Application No. 10-2016-7021778 mailed on Dec. 13, 2016, 4 pages. [cited by applicant]
Extended European Search Report issued in Application No. 16195834.3-1910 mailed on Nov. 23, 2016, 9 pages. [cited by applicant]
Japanese Office Action issued in Application No. 2016-561322, mailed on Feb. 5, 2018, 6 pages (with English translation). [cited by applicant]
Japanese Office Action for the related Application No. 2019-91428 dated May 11, 2020. [cited by applicant]
Japanese Office Action for the related Japanese Patent Application No. 2021-184030, mailed on Feb. 6, 2023, 6 pages (English translation). [cited by applicant]
USPTO. Office Action relating to U.S. Appl. No. 17/242,273, dated Nov. 10, 2022. [cited by applicant]
USPTO. Office Action relating to U.S. Appl. No. 18/150,561, dated Aug. 17, 2023. [cited by applicant]