IP Library Granted Patent US 12,315,512
Granted Patent B2
US 12,315,512 · App. 18/440,010 · Granted May 27, 2025

Virtual assistant identification of nearby computing devices

Inventor: Jian Wei Leong (San Francisco, CA)
Assignee: GOOGLE LLC
G10L15/30G06F3/165G10L15/22G10L2015/221G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,315,512
App. No.
18/440,010
Granted
May 27, 2025
Kind
B2
Abstract

In one example, a method includes method comprising: receiving audio data generated by a microphone of a current computing device; identifying, based on the audio data, one or more computing devices that each emitted a respective audio signal in response to speech reception being activated at the current computing device; and selecting either the current computing device or a particular computing device from the identified one or more computing devices to satisfy a spoken utterance determined based on the audio data.

Claims (61)

1. A battery powered device comprising:

one or more microphones,

one or more speakers;

one or more processors; and

memory storing instructions that, when executed, cause one or more of the processors to:

recognize speaking of a predetermined phrase in audio data received via one or more of the microphones;

in response to recognizing the speaking of the predetermined phrase:

transmit a request that:

causes a second computing device, that is located in a physical environment with the battery powered device, to emit a second device audio signal via a second device speaker component of the second computing device, and

causes a third computing device, that is also located in the physical environment, to emit a third device audio signal via a third device speaker component of the third computing device,

 wherein the second device audio signal has a second device emission frequency,

 wherein the third device audio signal has a third device emission frequency, and

 wherein the second device audio signal and the third device audio signal are each above a range of human hearing;

detect, at the battery powered device and via one or more of the microphones of the battery powered device, the second device audio signal and the third device audio signal;

determine, based on at least (i) the second device audio signal as detected at the battery powered device and (ii) the third device audio signal as detected at the battery powered device, that the second computing device is closer to to the battery powered device than is the third computing device; and

in response to determining that the second computing device is closer to the battery powered device than is the third computing device:

cause the second computing device to provide audible output.

2. The battery powered device of claim 1 , wherein in determining, based on at least (i) the second device audio signal as detected at the battery powered device and (ii) the third device audio signal as detected at the battery powered device, that the second computing device is closer to the battery powered device than is the third computing device, one or more of the processors are to:

determine that the second computing device is closer to the battery powered device than is the third computing device based on a second device audio signal strength of the second device audio signal as detected at the battery powered device and a third device audio signal strength of the third device audio signal as detected at the battery powered device.

3. The battery powered device of claim 2 , wherein the second device emission frequency differs from the third device emission frequency.

4. The battery powered device of claim 2 , wherein in causing the particular computing device to provide the audible output, one or more of the processors are to:

cause the particular computing device to play music as the audible output.

5. The battery powered device of claim 1 , wherein in transmitting the request, one or more of the processors are to:

transmit a second device request that causes the second computing device to emit the second device audio signal, and

transmit a third device request that causes the third computing device to emit the third device audio signal.

6. The battery powered device of claim 5 , wherein the second device request specifies one or more audio characteristics of the second device audio signal and wherein the third device request specifies one or more audio characteristics of the third device audio signal.

7. The battery powered device of claim 6 , wherein the one or more audio characteristics of the second device audio signal include the second device emission frequency for the second device audio signal, and wherein the one or more audio characteristics of the third device audio signal include the third device emission frequency for the third device audio signal.

8. The battery powered device of claim 7 , wherein the second device emission frequency differs from the third device emission frequency.

9. The battery powered device of claim 1 , wherein one or more of the processors, in executing the instructions, are further to:

determine that the second computing device is eligible for selection.

10. The battery powered device of claim 9 , wherein one or more of the processors, in executing the instructions, are further to:

determine that the third computing device is eligible for selection.

11. The battery powered device of claim 10 , wherein one or more of the processors, in executing the instructions, are further to:

determine that a fourth computing device is not eligible for selection.

12. A system comprising:

one or more processors; and

memory storing instructions that, when executed, cause one or more of the processors to:

activate speech reception at only a first computing device, wherein activating the speech reception at only the first computing device is in response to detecting speech reception input at only the first computing device;

receive audio data via one or more microphones of the first computing device, wherein receiving the audio data via the one or more microphones of the first computing device is in response to the speech reception being activated at the first computing device;

determine, based on the audio data, a spoken utterance that is captured in the audio data;

select, from a plurality of additional devices that can each satisfy the spoken utterance, a particular device for satisfying the spoken utterance, wherein in selecting the particular device one or more of the processors are to:

determine that, among the plurality of additional devices, the particular device is most proximal to the first computing device, and

select the particular device responsive to determining that the particular device is the most proximal to the first computing device, wherein proximity to the first computing device is utilized in the selecting responsive to speech reception being activated at the first computing device; and

in response to selecting the particular device:

cause the particular device to satisfy the spoken utterance.

13. The system of claim 12 , wherein one or more of the processors, in executing the instructions, are further to, at a later time that is subsequent to causing the particular device to satisfy the utterance:

again activate speech reception at only the first computing device in response to detecting the speech reception input at only the first computing device;

receive additional audio data via the one or more microphones of the first computing device, wherein in receiving the additional audio data via the one or more microphones of the first computing device one or more of the processors are to receive the additional audio data in response to the speech reception again being activated at the first computing device;

determine, based on the additional audio data, an additional spoken utterance that is captured in the additional audio data, wherein the additional spoken utterance is the same as the spoken utterance;

select, from the plurality of additional devices that can each satisfy the additional spoken utterance, an alternate particular device for satisfying the additional spoken utterance, wherein in selecting the alternate particular device one or more of the processors are to:

determine that, among the plurality of additional devices and at the later time, the alternate particular device is most proximal to the first computing device, and

select the alternate particular device responsive to determining that the alternate particular device is the most proximal to the first computing device, wherein proximity to the first computing device is utilized in the selecting responsive to speech reception again being activated at the first computing device; and

in response to selecting the alternate particular device:

cause the alternate particular device to satisfy the additional spoken utterance.

14. The system of claim 12 , wherein in causing the particular device to satisfy the spoken utterance one or more of the processors are to:

transmit, to the particular device, data that causes the particular device to satisfy the spoken utterance.

15. The system of claim 14 , wherein transmitting the data is via a Wi-Fi network, and wherein the first computing device and the particular device are both connected to the Wi-Fi network.

16. The system of claim 14 , wherein in causing the particular device to satisfy the spoken utterance one or more of the processors are to cause the particular device to produce output responsive to the spoken utterance.

17. The system of claim 16 , wherein the output comprises light.

18. The system of claim 16 , wherein in determining that the particular device is most proximal to the first computing device one or more of the processors are to determine that the particular device is most proximal to the first computing device based on analysis of an audio signal that is emitted by the particular device and that is captured in the audio data.

19. The system of claim 12 , wherein in determining that the particular device is most proximal to the first computing device one or more of the processors are to determine that the particular device is most proximal to the first computing device based on analysis of an audio signal that is emitted by the particular device and that is captured in the audio data.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 3, 2024
From: LEONG, JIAN WEI
To: GOOGLE INC.
Reel/Frame 066987/0795 →
CHANGE OF NAME Recorded Apr 3, 2024
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 066988/0079 →
Continuity (8)
Continuation 17856536 · Jul 1, 2022
Continuation 17688493 · Mar 7, 2022
Continuation 17506323 · Oct 20, 2021
Continuation 17397604 · Aug 9, 2021
Continuation 17201808 · Mar 15, 2021
Continuation 16409389 · May 10, 2019
Continuation 15355336 · Nov 18, 2016
Related Publication 20240185858A1 · Jun 6, 2024
References Cited (137)
US 5623652A · Vora et al. · 1997 [cited by applicant]
US 5625697A · Bowen et al. · 1997 [cited by applicant]
US 5867223A · Schindler et al. · 1999 [cited by applicant]
US 6038614A · Chan et al. · 2000 [cited by applicant]
US 6069621A · Schupack · 2000 [cited by applicant]
US 6185535B1 · Hedin et al. · 2001 [cited by applicant]
US 7184960B2 · Deisher et al. · 2007 [cited by applicant]
US 7830866B2 · Scherpbier · 2010 [cited by examiner]
US 8078508B1 · Rao · 2011 [cited by applicant]
US 8271287B1 · Kermani · 2012 [cited by applicant]
US 8627213B1 · Jouppi · 2014 [cited by applicant]
US 8903892B2 · Stinson et al. · 2014 [cited by applicant]
US 9098235B2 · Kuboki et al. · 2015 [cited by applicant]
US 9325853B1 · Marinov · 2016 [cited by examiner]
US 9491033B1 · Soyannwo · 2016 [cited by applicant]
US 9584653B1 · Lyren et al. · 2017 [cited by applicant]
US 9984686B1 · Mutagi et al. · 2018 [cited by applicant]
US 10002115B1 · Killian et al. · 2018 [cited by applicant]
US 10025447B1 · Dixit et al. · 2018 [cited by applicant]
US 10332523B2 · Leong · 2019 [cited by applicant]
US 10455277B2 · Shaw · 2019 [cited by applicant]
US 10571999B2 · Tanaka et al. · 2020 [cited by applicant]
US 10904624B2 · Vaysman et al. · 2021 [cited by applicant]
US 11087765B2 · Leong · 2021 [cited by applicant]
US 11227600B2 · Leong · 2022 [cited by applicant]
US 11270705B2 · Leong · 2022 [cited by applicant]
US 11380331B1 · Leong · 2022 [cited by applicant]
US 20030167335A1 · Alexander · 2003 [cited by applicant]
US 20060090132A1 · Jung · 2006 [cited by examiner]
US 20070055528A1 · Malyshev · 2007 [cited by applicant]
US 20080074546A1 · Momen · 2008 [cited by examiner]
US 20080225013A1 · Muylkens et al. · 2008 [cited by applicant]
US 20090204611A1 · Kamada et al. · 2009 [cited by applicant]
US 20090276223A1 · Jaiswal · 2009 [cited by applicant]
US 20100088100A1 · Lindahl · 2010 [cited by applicant]
US 20110091029A1 · Leblanc · 2011 [cited by applicant]
US 20110225531A1 · Irizarry · 2011 [cited by applicant]
US 20110257971A1 · Morrison · 2011 [cited by applicant]
US 20110283334A1 · Choi et al. · 2011 [cited by applicant]
US 20120034904A1 · LeBeau et al. · 2012 [cited by applicant]
US 20120084131A1 · Bergel et al. · 2012 [cited by applicant]
US 20120300958A1 · Klemmensen · 2012 [cited by examiner]
US 20120310396A1 · Ojanperä · 2012 [cited by applicant]
US 20130051572A1 · Goh · 2013 [cited by applicant]
US 20130091309A1 · Bjontegard et al. · 2013 [cited by applicant]
US 20130108076A1 · Chu et al. · 2013 [cited by applicant]
US 20130183944A1 · Mozer et al. · 2013 [cited by applicant]
US 20130275164A1 · Gruber et al. · 2013 [cited by applicant]
US 20140038669A1 · Shimizu et al. · 2014 [cited by applicant]
US 20140129231A1 · Herring et al. · 2014 [cited by applicant]
US 20140172140A1 · Buck · 2014 [cited by applicant]
US 20140278438A1 · Hart et al. · 2014 [cited by applicant]
US 20140304280A1 · Oursboum et al. · 2014 [cited by applicant]
US 20140304374A1 · McCue · 2014 [cited by examiner]
US 20140310643A1 · Karmanenko et al. · 2014 [cited by applicant]
US 20140325395A1 · Itakura · 2014 [cited by applicant]
US 20140331145A1 · Schoenefeld · 2014 [cited by applicant]
US 20150161998A1 · Park et al. · 2015 [cited by applicant]
US 20150172878A1 · Luna · 2015 [cited by applicant]
US 20150228274A1 · Leppanen et al. · 2015 [cited by applicant]
US 20150332431A1 · Tsuchida et al. · 2015 [cited by applicant]
US 20150370531A1 · Faaborg · 2015 [cited by applicant]
US 20150382047A1 · Van Os et al. · 2015 [cited by applicant]
US 20160050476A1 · Patil · 2016 [cited by applicant]
US 20160117793A1 · Sierra et al. · 2016 [cited by applicant]
US 20160155443A1 · Khan et al. · 2016 [cited by applicant]
US 20160179462A1 · Bjorkengren · 2016 [cited by applicant]
US 20160180853A1 · VanLund et al. · 2016 [cited by applicant]
US 20160260434A1 · Gelfenbeyn et al. · 2016 [cited by applicant]
US 20160277808A1 · Yu et al. · 2016 [cited by applicant]
US 20160301810A1 · Dimitroff et al. · 2016 [cited by applicant]
US 20160366084A1 · Malahy · 2016 [cited by applicant]
US 20160379638A1 · Basye et al. · 2016 [cited by applicant]
US 20170083285A1 · Meyers et al. · 2017 [cited by applicant]
US 20170199643A1 · Schnyder · 2017 [cited by examiner]
US 20170206808A1 · Engelke et al. · 2017 [cited by applicant]
US 20170251340A1 · Sanders et al. · 2017 [cited by applicant]
US 20170323633A1 · Lawrenson · 2017 [cited by examiner]
US 20170351481A1 · Raphael et al. · 2017 [cited by applicant]
US 20180061404A1 · Devaraj et al. · 2018 [cited by applicant]
US 20180061419A1 · Melendo Casado et al. · 2018 [cited by applicant]
US 20180092443A1 · Albers · 2018 [cited by applicant]
US 20180144748A1 · Leong · 2018 [cited by applicant]
US 20180253201A1 · High et al. · 2018 [cited by applicant]
US 20210044914A1 · Reilly et al. · 2021 [cited by applicant]
US 20210084431A1 · Reilly et al. · 2021 [cited by applicant]
US 20210201915A1 · Leong · 2021 [cited by applicant]
US 20210366487A1 · Leong · 2021 [cited by applicant]
US 20220044686A1 · Leong · 2022 [cited by applicant]
US 20220189488A1 · Leong · 2022 [cited by applicant]
US 20220335948A1 · Leong · 2022 [cited by applicant]
CN 104423576 · 2015 [cited by applicant]
CN 105051678 · 2015 [cited by applicant]
CN 105264485 · 2016 [cited by applicant]
EP 2298178 · 2011 [cited by applicant]
EP 2740413 · 2014 [cited by applicant]
EP 2932371 · 2015 [cited by applicant]
EP 2963630 · 2016 [cited by applicant]
JP 2003223188 · 2003 [cited by applicant]
JP 2007178934 · 2007 [cited by applicant]
JP 2016519805 · 2016 [cited by applicant]
JP 2017520008 · 2017 [cited by applicant]
KR 20100022297 · 2010 [cited by applicant]
KR 20110125725 · 2011 [cited by applicant]
KR 20150054703 · 2015 [cited by applicant]
KR 20160039244 · 2016 [cited by applicant]
WO 2016085776 · 2016 [cited by applicant]
Intellectual Property India; Extended Hearing Notice issued in Application No. 201927022517, 2 pages, dated Sep. 18, 2023. [cited by applicant]
Intellectual Property India; Hearing Notice issued in Application No. 201927022517, 2 pages, dated Aug. 3, 2023. [cited by applicant]
Deutsches Patent Office; Examination Report issued in Application No. 102017012415; 18 pages; dated Feb. 9, 2023. [cited by applicant]
Google Developers; Overview, Nearby Message API [online], [retrieved from the intemet Oct. 20, 2021] Retrieved from Internet: https://developers.google.com/nearby/messages/overview; 2 pages Oct. 20, 2021. [cited by applicant]
German Patent and Trademark Office; Examination Report issued in Application No. DE102017012396.9; 10 pages; dated Mar. 17, 2022. [cited by applicant]
The Korean Intellectual Property Office; Allowance of Patent issued in Application No. 10-2021-7021766, 3 pages, dated Mar. 28, 2022. [cited by applicant]
European Patent Office; International Search Report and Written Opinion issued in Application No. PCT/US2017/059342 dated Feb. 16, 2018. [cited by applicant]
United Kingdom Intellectual Property Office; Combined Search and Examination Report issued in Application No. 1715712.4 dated Mar. 23, 2018. [cited by applicant]
European Patent Office; International Report on Patentablity of Application No. PCT/US2017/059342; 17 pages; dated Oct. 15, 2018. [cited by applicant]
European Patent Office; Intention to Grant issued in App. No. 178050167; 43 pages; Sep. 16, 2019 1. [cited by applicant]
Japanese Patent Office; Notice of Allowance issued in Application No. 2019-526534 dated Dec. 2, 2019. [cited by applicant]
United Kingdom Intellectual Property Office; Intention to Grant issued in Application No. 1715712.4 dated Mar. 17, 2020. [cited by applicant]
Chinese Patent Office; Office Action issued for Application No. 201710901158 dated Jun. 16, 2020. [cited by applicant]
Issue 34348: Voice input should support Bluetooth microphones, Android Open Source Project-Issue Tracker, Jun. 2013, Retrieved from <https://code.google.com/p/android/issues/detail?id=34348> 8 pp. 2013. [cited by applicant]
LeFebvre, “Tell Siri Which Audio Input to Listen to [iOS Tips],” Cult of Mac, May 14, 2013, Retrieved from <http://www.cultofmac.com/227210/tell-siri-which-audio-input-to-listen-to-ios-tips/> 8 pp. 2013. [cited by applicant]
Exclusive: Google Will Soon Introduce ‘Nearby’ to Let Other 'People, Places, and Things' Know When You're Around, Jun. 6, 2014, Retrieved from http://www.androidpolice.com/2014/06/06/exclusive-google-will-soon-introduce… [cited by applicant]
Use Nearby to interact with what's around you—Accounts Help, Retrieved on Oct. 5, 2016 from https://support.google.com/accounts/answer/62602867hl=en, 5 pp. 2016. [cited by applicant]
Howard, “Activate Google Now “listening” via bluetooth headset,” Google Product Forms, Jun. 30, 2013, Retrieved from <https://productforums.google.com/forum/print/msg/websearch/-cTsRaVV5F4/jeSHROtqUqlj> 2 pp. 2013. [cited by applicant]
Korean Patent Office; Notice of Allowance issued in Application No. 1020197015790; 3 pages; dated Apr. 12, 2021. [cited by applicant]
Korean Patent Office; Notice of Office Action issued in Application No. 10-2019-7015790; 15 pages; dated Sep. 28, 2020. [cited by applicant]
China National Intellectual Property Administration; Notice of Allowance issued in Application No. 201710901158.X; 4 pages; dated May 11, 2020. [cited by applicant]
Intellectual Property India; Examination Report issued in Application No. 201927022517; 6 pages; dated Oct. 29, 2020. [cited by applicant]
Deutsches Patent Office; Office Action issued in Application No. 102017122182; 7 pages; dated Mar. 26, 2021. [cited by applicant]
U.S. Appl. No. 16/409,389; Decision on Appeal; Affirmed; 15 pages; dated Jun. 16, 2021. [cited by applicant]
Korean Patent Office; Notice of Office Action issued in Application No. 10-2021-7021766; 7 pages; dated Aug. 31, 2021. [cited by applicant]
Screen captures from YouTube video clip entitled “Chromecast—How to set up and use guest mode,” 13 pages, uploaded on Dec. 13, 2014 by user Robert Bevec. Retrieved from Internet: https://www.youtube.com/watch?v=pTDuKoUf… [cited by applicant]
Google Chrome Blog—posted by Jagjit Chawla; Chromecast, now with guest mode [online], [retrieved from the internet Oct. 20, 2021] Retrieved from Internet: https://chrome.googleblog.com/2014/12/chromecast-now-with-guest-… [cited by applicant]
Google Developers Blog—posted by Akshay Kannan; Connect with the World around You through Nearby APIs [online], [retrieved from the internet Oct. 20, 2021] Retrieved from Internet: https://developers.googleblog.com/2015… [cited by applicant]
Google Guest; Guest Mode; 4 pages [online], [retrieved from the internet Oct. 20, 2021] Retrieved from Internet: https://developers.google.com/cast/docs/guest_mode; last updated Dec. 8, 2020. [cited by applicant]
Urquidi, J.; OnHub Router from Google and TP-Link Sets to Simplify Wi-Fi Life [online], [retrieved from the internet Oct. 20, 2021] Retrieved from Internet: https://www.tomshardware.com/news/google-tplink-onhub-router,2… [cited by applicant]