IP Library › Granted Patent US 12,253,620
Granted Patent B2
US 12,253,620 · App. 17/449,054 · Granted Mar 18, 2025

Multi-user intelligent assistance

Inventors: Kazuhito Koishida (Redmond, WA); Alexander A. Popov (Kirkland, WA); Uros Batricevic (Redmond, WA); Steven Nabil Bathiche (Bellevue, WA)
Assignee: Microsoft Technology Licensing, LLC
G01S5/28A61B5/0205A61B5/0507A61B5/1113A61B5/117A61B5/7475G01S5/18G01S13/726G06F1/3206G06F1/3231G06F1/324G06F3/011G06F3/017G06F3/0304G06F3/0482G06F3/04842G06F3/167G06F18/251G06F18/28G06F18/29G06F18/41G06F21/32G06F21/35G06F40/211G06F40/35G06N5/025G06N5/047G06N20/00G06T7/248G06T7/292G06T7/60G06T7/70G06T7/74G06V10/94G06V20/40G06V20/52G06V20/653G06V30/274G06V40/10G06V40/103G06V40/166G06V40/167G06V40/172G06V40/173G06V40/23G06V40/25G07C9/28G08B13/1427G10L15/02G10L15/063G10L15/08G10L15/18G10L15/1815G10L15/1822G10L15/19G10L15/22G10L15/24G10L15/26G10L15/28G10L15/32G10L17/04G10L17/08G10L25/51H04L51/02H04L63/102H04L67/12H04L67/535H04N7/181H04N7/188H04N21/231H04N21/42203H04N21/44218H04N21/44224H04N23/11H04N23/611H04R1/406H04R3/005H04W4/029H04W4/33A61B5/05A61B5/1118G01S5/16G01S11/14G01S13/38G01S13/867G01S13/888G06F3/0488G06F16/70G06F2203/0381G06F2221/2111G06N3/044G06T2207/10016G06T2207/10024G06T2207/20101G06T2207/30196G06T2207/30201G06T2207/30204G06T2207/30232G06V2201/05G07C9/32G08B29/186G10L2015/0635G10L2015/088G10L2015/223G10L2015/225G10L2015/228G10L17/00H04N23/90Y02D10/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,253,620
App. No.
17/449,054
Filed
Sep 27, 2021
Granted
Mar 18, 2025
Kind
B2
Art Unit
2658
USPC
704/251
Abstract

An intelligent assistant records speech spoken by a first user and determines a self-selection score for the first user. The intelligent assistant sends the self-selection score to another intelligent assistant, and receives a remote-selection score for the first user from the other intelligent assistant. The intelligent assistant compares the self-selection score to the remote-selection score. If the self-selection score is greater than the remote-selection score, the intelligent assistant responds to the first user and blocks subsequent responses to all other users until a disengagement metric of the first user exceeds a blocking threshold. If the self-selection score is less than the remote-selection score, the intelligent assistant does not respond to the first user.

Claims (51)

1. An intelligent assistant computer, comprising:

a logic machine; and

a storage machine holding instructions executable by the logic machine to:

recognize another intelligent assistant computer;

record speech spoken by a first user;

determine a self-selection score for the first user based on the speech spoken by the first user;

receive a remote-selection score for the first user from the other intelligent assistant computer;

if the self-selection score is greater than the remote-selection score, respond to the first user, determine a disengagement metric of the first user based on recorded speech spoken by the first user, and block subsequent responses to all other users until the disengagement metric of the first user exceeds a blocking threshold;

if the self-selection score is less than the remote-selection score, do not respond to the first user; and

stop blocking subsequent responses to another user responsive to a new self-selection score for the first user being less than a new remote-selection score for the first user.

2. The intelligent assistant computer of claim 1 , wherein the self-selection score is determined based further on a signal-to-noise ratio of recorded speech spoken by the first user.

3. The intelligent assistant computer of claim 1 , wherein the self-selection score is determined based further on an amplitude of the recorded speech.

4. The intelligent assistant computer of claim 1 , wherein the self-selection score is determined based further on a user identification confidence value indicating a likelihood that the first user is an identified user.

5. The intelligent assistant computer of claim 1 , wherein the self-selection score is determined based further on a proximity of the first user to one or more microphones of the intelligent assistant computer.

6. The intelligent assistant computer of claim 1 , wherein the self-selection score is determined based further on image data collected by one or more cameras of the intelligent assistant computer.

7. The intelligent assistant computer of claim 1 , wherein the storage machine further holds instructions executable to:

determine that the other user is unidentified; and

responsive to determining that the other user is unidentified, lower the blocking threshold.

8. The intelligent assistant computer of claim 1 , wherein the storage machine further holds instructions executable to increase the disengagement metric as a time since a last recorded speech spoken by the first user increases.

9. The intelligent assistant computer of claim 1 , wherein the storage machine further holds instructions executable to send the self-selection score and future self-selection scores to the other intelligent assistant computer, and to dynamically vary a frequency at which the intelligent assistant computer sends the future self-selection scores to the other intelligent assistant computer.

10. The intelligent assistant computer of claim 1 , wherein the storage machine further holds instructions executable to adjust the blocking threshold based on one or more of image data collected by one or more cameras of the intelligent assistant computer, eye gaze data collected by the one or more cameras, gesture data, and audio data collected by one or more microphones of the intelligent assistant computer.

11. The intelligent assistant computer of claim 1 , wherein the intelligent assistant computer is operatively coupled to two or more audio output devices, and wherein the storage machine further holds instructions executable to identify a selected audio output device from the two or more audio output devices with which to respond to the first user, the selected audio output device recognized as being closest among the two or more audio output devices to the first user.

12. The intelligent assistant computer of claim 1 , wherein the storage machine further holds instructions executable to stop blocking subsequent responses to the other user responsive further to receiving a command from the first user instructing the intelligent assistant computer to respond to the other user.

13. The intelligent assistant computer of claim 12 , wherein the command includes a request for the intelligent assistant computer to listen for a request from the other user.

14. The intelligent assistant computer of claim 12 , wherein the command includes a request for the intelligent assistant compute to identify the other user.

15. The intelligent assistant computer of claim 1 , wherein the storage machine further holds instructions executable to retain a conversation with the first user at the intelligent assistant computer responsive to the remote-selection score being higher than the self-selection score, and further responsive to another conversation with the other intelligent assistant computer being conducted.

16. At an intelligent assistant computer, a method, comprising:

recognizing another intelligent assistant computer;

recording speech spoken by a first user;

obtaining a self-selection score for the first user based on the speech spoken by the first user;

obtaining a remote-selection score for the first user from the other intelligent assistant computer;

if the self-selection score is greater than the remote-selection score, responding to the first user, determining a disengagement metric of the first user based on recorded speech spoken by the first user, and blocking subsequent responses to all other users until the disengagement metric of the first user exceeds a blocking threshold;

if the self-selection score is less than the remote-selection score, not responding to the first user; and

stopping blocking subsequent responses to another user responsive to a new self-selection score for the first user being less than a new remote-selection score for the first user.

17. The method of claim 16 , wherein the self-selection score and the remote-selection score are obtained from one or more remote computing systems remote from the intelligent assistant computer and the other intelligent assistant computer.

18. The method of claim 16 , further comprising:

determining that the other user is unidentified; and

responsive to determining that the other user is unidentified, lowering the blocking threshold.

19. A computing system, comprising:

a logic machine; and

a storage machine holding instructions executable by the logic machine to:

receive, from a first intelligent assistant computer, data representing speech spoken by a first user;

receive, from a second intelligent assistant computer, data representing speech spoken by the first user;

determine a selection score for the first intelligent assistant computer based on the data received from the first intelligent assistant computer;

determine a selection score for the second intelligent assistant computer based on the data received from the second intelligent assistant computer;

select the first intelligent assistant computer as a responding intelligent assistant computer if the selection score for the first intelligent assistant computer is greater than the selection score for the second intelligent assistant computer;

select the second intelligent assistant computer as the responding intelligent assistant computer if the selection score for the second intelligent assistant computer is greater than the selection score for the first intelligent assistant computer;

send an instruction to the responding intelligent assistant computer to respond to the first user;

determine a disengagement metric of the first user; and

send an instruction to the responding intelligent assistant computer to block subsequent responses to all other users until the disengagement metric of the first user exceeds a blocking threshold.

20. The computing system of claim 19 , wherein the storage machine further holds instructions executable to determine respective selection scores for one or more other intelligent assistant computers.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 27, 2021
From: KOISHIDA, KAZUHITO; POPOV, ALEXANDER A.; BATRICEVIC, UROS; BATHICHE, STEVEN NABIL
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 057613/0920 →
Continuity (4)
Continuation 15657822 · Jul 24, 2017
Provisional Application 62482165 · Apr 5, 2017
Provisional Application 62459020 · Feb 14, 2017
Related Publication 20220012470A1 · Jan 13, 2022
References Cited (384)
US 5644735A · Luciw et al. · 1997 [cited by applicant]
US 5784568A · Needham · 1998 [cited by applicant]
US 5794189A · Gould · 1998 [cited by applicant]
US 6148280A · Kramer · 2000 [cited by applicant]
US 6163699A · Naor et al. · 2000 [cited by applicant]
US 6502082B1 · Toyama et al. · 2002 [cited by applicant]
US 6535876B2 · Zhai · 2003 [cited by applicant]
US 6567778B1 · Chao Chang et al. · 2003 [cited by applicant]
US 6587851B1 · Ditcharo et al. · 2003 [cited by applicant]
US 6711543B2 · Cameron et al. · 2004 [cited by applicant]
US 7016849B2 · Arnold et al. · 2006 [cited by applicant]
US 7027985B2 · Wutte · 2006 [cited by applicant]
US 7111980B2 · Pavlidis et al. · 2006 [cited by applicant]
US 7120865B1 · Horvitz et al. · 2006 [cited by applicant]
US 7181500B2 · Jen et al. · 2007 [cited by applicant]
US 7209895B2 · Kundtz et al. · 2007 [cited by applicant]
US 7263663B2 · Ballard et al. · 2007 [cited by applicant]
US 7454331B2 · Vinton et al. · 2008 [cited by applicant]
US 7474889B2 · Bhakta et al. · 2009 [cited by applicant]
US 7483899B2 · Berry et al. · 2009 [cited by applicant]
US 7603427B1 · Horvitz et al. · 2009 [cited by applicant]
US 7647228B2 · Silvera et al. · 2010 [cited by applicant]
US 7697922B2 · Mcquaide et al. · 2010 [cited by applicant]
US 7844666B2 · Baribault et al. · 2010 [cited by applicant]
US 7920678B2 · Cooper et al. · 2011 [cited by applicant]
US 7962360B2 · Mechaley et al. · 2011 [cited by applicant]
US 8000768B2 · Zhao et al. · 2011 [cited by applicant]
US 8015006B2 · Kennewick · 2011 [cited by applicant]
US 8099287B2 · Bodin et al. · 2012 [cited by applicant]
US 8155968B2 · Sugiyama et al. · 2012 [cited by applicant]
US 8175530B2 · Kane et al. · 2012 [cited by applicant]
US 8219406B2 · Yu et al. · 2012 [cited by applicant]
US 8223932B2 · Forbes et al. · 2012 [cited by applicant]
US 8250157B2 · Millmore et al. · 2012 [cited by applicant]
US 8358599B2 · Shaffer et al. · 2013 [cited by applicant]
US 8380514B2 · Bodin et al. · 2013 [cited by applicant]
US 8391455B2 · Kelkar et al. · 2013 [cited by applicant]
US 8395653B2 · Feng et al. · 2013 [cited by applicant]
US 8515765B2 · Baldwin et al. · 2013 [cited by applicant]
US 8542276B2 · Huang et al. · 2013 [cited by applicant]
US 8542675B2 · Cormier et al. · 2013 [cited by applicant]
US 8543407B1 · Gagnon et al. · 2013 [cited by applicant]
US 8626511B2 · Lebeau et al. · 2014 [cited by applicant]
US 8639401B2 · Bailey et al. · 2014 [cited by applicant]
US 8676581B2 · Flaks et al. · 2014 [cited by applicant]
US 8688453B1 · Joshi et al. · 2014 [cited by applicant]
US 8707201B1 · Aradhye et al. · 2014 [cited by applicant]
US 8731902B2 · Beld et al. · 2014 [cited by applicant]
US 8731929B2 · Kennewick et al. · 2014 [cited by applicant]
US 8799000B2 · Guzzoni et al. · 2014 [cited by applicant]
US 8825510B2 · Hamilton · 2014 [cited by applicant]
US 8836643B2 · Romera Joliff et al. · 2014 [cited by applicant]
US 8849670B2 · Di Cristo et al. · 2014 [cited by applicant]
US 8954318B2 · Barve et al. · 2015 [cited by applicant]
US 8965765B2 · Zweig et al. · 2015 [cited by applicant]
US 8983383B1 · Haskin · 2015 [cited by applicant]
US 9015049B2 · Baldwin et al. · 2015 [cited by applicant]
US 9083816B2 · Gunderson et al. · 2015 [cited by applicant]
US 9098467B1 · Blanksteen et al. · 2015 [cited by applicant]
US 9148625B2 · Kleinsteiber et al. · 2015 [cited by applicant]
US 9154730B2 · Lee et al. · 2015 [cited by applicant]
US 9160730B2 · Sheller et al. · 2015 [cited by applicant]
US 9161172B2 · Poduri et al. · 2015 [cited by applicant]
US 9165566B2 · Klein et al. · 2015 [cited by applicant]
US 9179256B1 · Pietraniec · 2015 [cited by applicant]
US 9190075B1 · Cronin et al. · 2015 [cited by applicant]
US 9202466B2 · Heckmann · 2015 [cited by applicant]
US 9214157B2 · Bangalore · 2015 [cited by applicant]
US 9245036B2 · Poornachandran et al. · 2016 [cited by applicant]
US 9299350B1 · Dumont et al. · 2016 [cited by applicant]
US 9311915B2 · Weinstein et al. · 2016 [cited by applicant]
US 9317113B1 · Karakotsios et al. · 2016 [cited by applicant]
US 9330658B2 · Bezar · 2016 [cited by applicant]
US 9373322B2 · Goussard · 2016 [cited by applicant]
US 9386359B2 · Adler et al. · 2016 [cited by applicant]
US 9391986B2 · Schultz et al. · 2016 [cited by applicant]
US 9400902B2 · Schoner et al. · 2016 [cited by applicant]
US 9401147B2 · Jitkoff et al. · 2016 [cited by applicant]
US 9405427B2 · Curtis et al. · 2016 [cited by applicant]
US 9412363B2 · Hakkani-Tur et al. · 2016 [cited by applicant]
US 9437206B2 · Yu et al. · 2016 [cited by applicant]
US 9449229B1 · Laska et al. · 2016 [cited by applicant]
US 9461972B1 · Mehta · 2016 [cited by applicant]
US 9465833B2 · Venkataraman et al. · 2016 [cited by applicant]
US 9477643B2 · Barve et al. · 2016 [cited by applicant]
US 9495957B2 · Weider et al. · 2016 [cited by applicant]
US 9521364B2 · Huitema et al. · 2016 [cited by applicant]
US 9548050B2 · Gruber et al. · 2017 [cited by applicant]
US 9552350B2 · Miller et al. · 2017 [cited by applicant]
US 9565284B2 · Cheatham et al. · 2017 [cited by applicant]
US 9585563B2 · Mensinger et al. · 2017 [cited by applicant]
US 10255269B2 · Quirk et al. · 2019 [cited by applicant]
US 10460215B2 · Herold et al. · 2019 [cited by applicant]
US 10467509B2 · Albadawi et al. · 2019 [cited by applicant]
US 10467510B2 · Albadawi et al. · 2019 [cited by applicant]
US 10496905B2 · Solomon et al. · 2019 [cited by applicant]
US 10565989B1 · Wheeler et al. · 2020 [cited by applicant]
US 10579912B2 · Holtmann · 2020 [cited by applicant]
US 10621478B2 · Albadawi et al. · 2020 [cited by applicant]
US 10628714B2 · Pradeep et al. · 2020 [cited by applicant]
US 10765954B2 · Malyuk · 2020 [cited by applicant]
US 10817760B2 · Pradeep et al. · 2020 [cited by applicant]
US 10824921B2 · Bathiche et al. · 2020 [cited by applicant]
US 10957311B2 · Solomon et al. · 2021 [cited by applicant]
US 10984782B2 · Finkelstein et al. · 2021 [cited by applicant]
US 11010601B2 · Bathiche et al. · 2021 [cited by applicant]
US 11017765B2 · Solomon et al. · 2021 [cited by applicant]
US 11100384B2 · Finkelstein et al. · 2021 [cited by applicant]
US 11126825B2 · Herold et al. · 2021 [cited by applicant]
US 20030033152A1 · Cameron · 2003 [cited by applicant]
US 20030046401A1 · Abbott · 2003 [cited by applicant]
US 20030135382A1 · Marejka et al. · 2003 [cited by applicant]
US 20030154398A1 · Eaton et al. · 2003 [cited by applicant]
US 20030157968A1 · Boman et al. · 2003 [cited by applicant]
US 20030233230A1 · Ammicht et al. · 2003 [cited by applicant]
US 20050108074A1 · Bloechl et al. · 2005 [cited by applicant]
US 20050182619A1 · Azara et al. · 2005 [cited by applicant]
US 20060143216A1 · Gupta et al. · 2006 [cited by applicant]
US 20060235688A1 · Bicego et al. · 2006 [cited by applicant]
US 20060270385A1 · Morris · 2006 [cited by applicant]
US 20070047708A1 · Boillot et al. · 2007 [cited by applicant]
US 20070133437A1 · Wengrovitz et al. · 2007 [cited by applicant]
US 20070192910A1 · Vu et al. · 2007 [cited by applicant]
US 20070263823A1 · Jalava et al. · 2007 [cited by applicant]
US 20080004881A1 · Attwater et al. · 2008 [cited by applicant]
US 20080091406A1 · Baldwin et al. · 2008 [cited by applicant]
US 20080270131A1 · Fukuda et al. · 2008 [cited by applicant]
US 20080319733A1 · Pulz et al. · 2008 [cited by applicant]
US 20090018835A1 · Cooper et al. · 2009 [cited by applicant]
US 20090022368A1 · Matsuoka et al. · 2009 [cited by applicant]
US 20090094555A1 · Viitala · 2009 [cited by applicant]
US 20090117922A1 · Bell et al. · 2009 [cited by applicant]
US 20090177990A1 · Chen et al. · 2009 [cited by applicant]
US 20100121973A1 · Lobacheva et al. · 2010 [cited by applicant]
US 20110112837A1 · Kurki-Suonio et al. · 2011 [cited by applicant]
US 20110159884A1 · Chawla · 2011 [cited by applicant]
US 20110314477A1 · Zhong · 2011 [cited by applicant]
US 20120245925A1 · Guha et al. · 2012 [cited by applicant]
US 20120274464A1 · Sweeney et al. · 2012 [cited by applicant]
US 20120323933A1 · He et al. · 2012 [cited by applicant]
US 20130036367A1 · Deroos et al. · 2013 [cited by applicant]
US 20130073293A1 · Jang et al. · 2013 [cited by applicant]
US 20130141587A1 · Petricoin, Jr. · 2013 [cited by applicant]
US 20130144616A1 · Bangalore · 2013 [cited by applicant]
US 20130222226A1 · Terlouw · 2013 [cited by applicant]
US 20130250035A1 · Murali et al. · 2013 [cited by applicant]
US 20130304476A1 · Kim et al. · 2013 [cited by applicant]
US 20130342672A1 · Gray et al. · 2013 [cited by applicant]
US 20130346511A1 · Park et al. · 2013 [cited by applicant]
US 20140070945A1 · Dave et al. · 2014 [cited by applicant]
US 20140074472A1 · Lin et al. · 2014 [cited by applicant]
US 20140075349A1 · Yun et al. · 2014 [cited by applicant]
US 20140129661A1 · Thyagaraja · 2014 [cited by applicant]
US 20140139616A1 · Pinter et al. · 2014 [cited by applicant]
US 20140172953A1 · Blanksteen · 2014 [cited by examiner]
US 20140195247A1 · Parkinson et al. · 2014 [cited by applicant]
US 20140222430A1 · Rao · 2014 [cited by applicant]
US 20140236594A1 · Clarke et al. · 2014 [cited by applicant]
US 20140244267A1 · Levi · 2014 [cited by applicant]
US 20140249817A1 · Hart et al. · 2014 [cited by applicant]
US 20140258324A1 · Mauro et al. · 2014 [cited by applicant]
US 20140267034A1 · Krulce et al. · 2014 [cited by applicant]
US 20140278413A1 · Pitschel et al. · 2014 [cited by applicant]
US 20140282003A1 · Gruber et al. · 2014 [cited by applicant]
US 20140282703A1 · Garg · 2014 [cited by applicant]
US 20140282893A1 · Sheller et al. · 2014 [cited by applicant]
US 20140288934A1 · Kennewick et al. · 2014 [cited by applicant]
US 20140297284A1 · Gruber et al. · 2014 [cited by applicant]
US 20140297802A1 · Liang · 2014 [cited by applicant]
US 20140310001A1 · Kalns et al. · 2014 [cited by applicant]
US 20140325402A1 · Jung et al. · 2014 [cited by applicant]
US 20140365209A1 · Evermann · 2014 [cited by applicant]
US 20140372344A1 · Morris et al. · 2014 [cited by applicant]
US 20140379296A1 · Nathan et al. · 2014 [cited by applicant]
US 20150011842A1 · Steinberg-Shapira · 2015 [cited by applicant]
US 20150019210A1 · Bangalore et al. · 2015 [cited by applicant]
US 20150058017A1 · Singh et al. · 2015 [cited by applicant]
US 20150066447A1 · Lin et al. · 2015 [cited by applicant]
US 20150088518A1 · Kim et al. · 2015 [cited by applicant]
US 20150088814A1 · Van Der Laan et al. · 2015 [cited by applicant]
US 20150095030A1 · Andrew et al. · 2015 [cited by applicant]
US 20150143406A1 · Cho et al. · 2015 [cited by applicant]
US 20150154002A1 · Weinstein et al. · 2015 [cited by applicant]
US 20150154291A1 · Shepherd et al. · 2015 [cited by applicant]
US 20150189100A1 · Wang · 2015 [cited by applicant]
US 20150199965A1 · Leak et al. · 2015 [cited by applicant]
US 20150200892A1 · Ramachandran · 2015 [cited by applicant]
US 20150228276A1 · Baldwin et al. · 2015 [cited by applicant]
US 20150254061A1 · Gelfenbeyn et al. · 2015 [cited by applicant]
US 20150261293A1 · Thukral et al. · 2015 [cited by applicant]
US 20150281162A1 · Shen et al. · 2015 [cited by applicant]
US 20150301796A1 · Visser et al. · 2015 [cited by applicant]
US 20150314454A1 · Breazeal et al. · 2015 [cited by applicant]
US 20150340040A1 · Mun et al. · 2015 [cited by applicant]
US 20160034424A1 · Won · 2016 [cited by applicant]
US 20160055240A1 · Tur et al. · 2016 [cited by applicant]
US 20160063692A1 · Divakaran · 2016 [cited by applicant]
US 20160077794A1 · Kim et al. · 2016 [cited by applicant]
US 20160080165A1 · Ehsani et al. · 2016 [cited by applicant]
US 20160154787A1 · Allen et al. · 2016 [cited by applicant]
US 20160154792A1 · Xu et al. · 2016 [cited by applicant]
US 20160155442A1 · Kannan · 2016 [cited by applicant]
US 20160170710A1 · Kim et al. · 2016 [cited by applicant]
US 20160189717A1 · Kannan et al. · 2016 [cited by applicant]
US 20160189733A1 · Vasilieff et al. · 2016 [cited by applicant]
US 20160216117A9 · Bandyopadhyay et al. · 2016 [cited by applicant]
US 20160217128A1 · Baum et al. · 2016 [cited by applicant]
US 20160259419A1 · Chatterjee · 2016 [cited by applicant]
US 20160292536A1 · Irie et al. · 2016 [cited by applicant]
US 20160307413A1 · Rafii et al. · 2016 [cited by applicant]
US 20160320838A1 · Teller et al. · 2016 [cited by applicant]
US 20160321617A1 · Shastri et al. · 2016 [cited by applicant]
US 20160330042A1 · Andersen · 2016 [cited by applicant]
US 20160345406A1 · Donhowe et al. · 2016 [cited by applicant]
US 20160351196A1 · Fanty · 2016 [cited by applicant]
US 20160358092A1 · Kurpinski et al. · 2016 [cited by applicant]
US 20160364382A1 · Sarikaya · 2016 [cited by applicant]
US 20160370879A1 · Sharma · 2016 [cited by applicant]
US 20160373269A1 · Okubo et al. · 2016 [cited by applicant]
US 20160379172A1 · Song et al. · 2016 [cited by applicant]
US 20170000392A1 · Smith · 2017 [cited by applicant]
US 20170004828A1 · Lee et al. · 2017 [cited by applicant]
US 20170048343A1 · Arabo et al. · 2017 [cited by applicant]
US 20170060521A1 · Cellier et al. · 2017 [cited by applicant]
US 20170061316A1 · Tur et al. · 2017 [cited by applicant]
US 20170076720A1 · Gopalan · 2017 [cited by examiner]
US 20170083285A1 · Meyers · 2017 [cited by examiner]
US 20170351749A1 · Quirk et al. · 2017 [cited by applicant]
US 20180144748A1 · Leong · 2018 [cited by examiner]
US 20180231653A1 · Pradeep et al. · 2018 [cited by applicant]
US 20180232201A1 · Holtmann · 2018 [cited by applicant]
US 20180232563A1 · Albadawi et al. · 2018 [cited by applicant]
US 20180232571A1 · Bathiche et al. · 2018 [cited by applicant]
US 20180232608A1 · Pradeep et al. · 2018 [cited by applicant]
US 20180232645A1 · Finkelstein et al. · 2018 [cited by applicant]
US 20180232662A1 · Solomon et al. · 2018 [cited by applicant]
US 20180232902A1 · Albadawi et al. · 2018 [cited by applicant]
US 20180233132A1 · Herold et al. · 2018 [cited by applicant]
US 20180233139A1 · Finkelstein et al. · 2018 [cited by applicant]
US 20180233140A1 · Koishida et al. · 2018 [cited by applicant]
US 20180233141A1 · Solomon et al. · 2018 [cited by applicant]
US 20180233142A1 · Koishida et al. · 2018 [cited by applicant]
US 20180233145A1 · Bathiche et al. · 2018 [cited by applicant]
US 20180260680A1 · Finkelstein et al. · 2018 [cited by applicant]
US 20180293221A1 · Finkelstein et al. · 2018 [cited by applicant]
US 20180361258A1 · Malyuk · 2018 [cited by applicant]
US 20190236416A1 · Wang et al. · 2019 [cited by applicant]
US 20200012906A1 · Albadawi et al. · 2020 [cited by applicant]
US 20200042839A1 · Herold et al. · 2020 [cited by applicant]
US 20200104653A1 · Solomon et al. · 2020 [cited by applicant]
CA 2408584A1 · 2001 [cited by applicant]
CA 2787351A1 · 2011 [cited by applicant]
CN 1306271A · 2001 [cited by applicant]
CN 101894553A · 2010 [cited by applicant]
CN 102510426A · 2012 [cited by applicant]
CN 103761505A · 2014 [cited by applicant]
CN 104423563A · 2015 [cited by applicant]
CN 104462175A · 2015 [cited by applicant]
CN 105575392A · 2016 [cited by applicant]
CN 105810194A · 2016 [cited by applicant]
CN 105917404A · 2016 [cited by applicant]
CN 106062869A · 2016 [cited by applicant]
CN 107000210A · 2017 [cited by applicant]
EP 2671411A1 · 2013 [cited by applicant]
JP 2016099575A · 2016 [cited by applicant]
KR 20010057768A · 2001 [cited by applicant]
NL 2012491A · 2015 [cited by applicant]
WO 0109755A2 · 2001 [cited by applicant]
WO 03093963A2 · 2003 [cited by applicant]
WO 2008103206A1 · 2008 [cited by applicant]
WO 2013173511A2 · 2013 [cited by applicant]
WO 2014026636A1 · 2014 [cited by applicant]
WO 2014059342A2 · 2014 [cited by applicant]
WO 2014137751A1 · 2014 [cited by applicant]
WO 2014197334A2 · 2014 [cited by applicant]
WO 2015047557A1 · 2015 [cited by applicant]
WO 2015179510A1 · 2015 [cited by applicant]
WO 2016057268A1 · 2016 [cited by applicant]
WO 2016111881A1 · 2016 [cited by applicant]
“Office Action Issued in Indian Patent Application No. 201947029579”, Mailed Date: Jan. 4, 2022, 6 Pages. [cited by applicant]
“Office Action Issued in Indian Patent Application No. 201947030165”, Mailed Date: Jan. 5, 2022, 7 Pages. [cited by applicant]
“Final Office Action Issued in U.S. Appl. No. 16/005,470”, Mailed Date: Apr. 6, 2023, 33 Pages. [cited by applicant]
“Office Action Issued in Chinese Patent Application No. 201880011965.7”, Mailed Date: Mar. 25, 2023, 6 Pages. [cited by applicant]
“Office Action and Search Report Issued in Chinese Patent Application No. 201880011917.8”, Mailed Date: Apr. 7, 2023, 12 Pages. [cited by applicant]
“Non Final Office Action Issued in U.S. Appl. No. 16/005,470”, Mailed Date: Oct. 21, 2022, 28 Pages. [cited by applicant]
“Office Action and Search Report Issued in Chinese Patent Application No. 201880011917.8”, Mailed Date: Nov. 3, 2022, 12 Pages. [cited by applicant]
“Office Action Issued in European Patent Application No. 19714954.5”, Mailed Date: Dec. 23, 2022, 5 Pages. [cited by applicant]
“Summons to Attend Oral Proceedings Issued in European Patent Application No. 18706370.6”, Mailed Date: Feb. 9, 2023, 5 Pages. [cited by applicant]
“Extended European Search Report issued in European Patent Application No. 22153942.2”, Mailed Date: May 10, 2022, 16 Pages. [cited by applicant]
“Office Action issued in European Patent Application No. 19727161.2”, Mailed Date: Jun. 2, 2022, 6 Pages. [cited by applicant]
“Final Office Action Issued in U.S. Appl. No. 16/005,470”, Mailed Date: Jul. 20, 2022, 33 Pages. [cited by applicant]
“First Office Action and Search Report Issued in Chinese Patent Application No. 201880011965.7”, Mailed Date: Aug. 31, 2022, 24 Pages. [cited by applicant]
“Non Final Office Action Issued in U.S. Appl. No. 16/005,470”, Mailed Date : Mar. 16, 2022, 19 Pages. [cited by applicant]
“Non Final Office Action Issued in U.S. Appl. No. 16/005,470”, Mailed Date: Apr. 5, 2022, 21 Pages. [cited by applicant]
“Office Action Issued in Indian Patent Application No. 201947029577”, Mailed Date: Oct. 8, 2021, 6 Pages. [cited by applicant]
“Office Action Issued in European Patent Application No. 18707798.7”, Mailed Date: Oct. 20, 2021, 6 Pages. [cited by applicant]
“Notice of Allowance Issued in European Patent Application No. 18707798.7”, Mailed Date: Oct. 12, 2023, 3 Pages. [cited by applicant]
“Intelligent Customer service Robot,” Customer Care & Management World, Aug. 31, 2017, 16 pages. [cited by applicant]
Second Office Action Received for Chinese Application No. 201980022427.2, mailed on Dec. 22, 2023, 34 pages. [cited by applicant]
Office Action Received for Chinese Application No. 202111348785.8, mailed on Nov. 11, 2023, 10 pages (English Translation Provided). [cited by applicant]
“Foveated imaging”, accessed on link https://en.wikipedia.org/wiki/Foveated_imaging, 2024, 4 pages. [cited by applicant]
Communication 71(3) Received for European Application No. 19727161.2, mailed on Apr. 5, 2024, 7 pages. [cited by applicant]
Communication 94(3) Received for European Application No. 19714954.5, mailed on Apr. 19, 2024, 5 pages. [cited by applicant]
Office Action Received for Chinese Application No. 202111348785.8, mailed on Mar. 26, 2024, 05 pages (English Translation Provided). [cited by applicant]
“Notice of Allowance Issued in European Patent Application No. 18707798.7”, Mailed Date: Jul. 12, 2023, 8 Pages. [cited by applicant]
“Notice of Allowance Issued in Chinese Patent Application No. 201880011965.7”, Mailed Date: Aug. 4, 2023, 8 Pages. [cited by applicant]
“Office Action Issued in European Patent Application No. 18707798.7”, Mailed Date: Jun. 7, 2023, 4 Pages. [cited by applicant]
“Office Action Issued in Chinese Patent Application No. 201880011917.8”, Mailed Date: Jun. 21, 2023, 6 Pages. [cited by applicant]
“Office Action Issued in European Patent Application No. 19727161.2”, Mailed Date: Apr. 25, 2023, 9 Pages. [cited by applicant]
Bhargava., et al., “Easy Contextual Intent Prediction and Slot Detection”, In IEEE International Conference on Acoustics, Speech and Signal Processing, 2013, 5 Pages. [cited by applicant]
Bucher., et al., “Anything to Clarify? Report your Parsing Ambiguities!,” In Proceedings of the 15th European Conference on Artificial Intelligence, 2002, 5 Pages. [cited by applicant]
Carvalho., et al., “Analysis of Heart Rate Monitoring Using a Webcam,” In International Journal of Advanced Research in Computer and Communication Engineering, May 2014, vol. 3, Issue 5, pp. 6593-6595. [cited by applicant]
Celikyilmaz., et al., Context Memory Networks for Multi-Objective Semantic Parsing in Conversational Understanding, Retrieved From: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/AttentionMemory_Cel… [cited by applicant]
Chen., et al., “Distant Multi-Speaker Voice Activity Detection Using Relative Energy Rati,” Retrieved on: http://www.handicams-fet.eu/uploads/files/27.pdf, 2017, 11 Pages. [cited by applicant]
Chen., et al., “Improving Pronominal and Deictic Co-Reference Resolution with Multi-Modal Features”, In Proceedings of 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, 2011, pp. 307-311. [cited by applicant]
Decision to Grant Issued in European Patent Application No. 18707800.1, mailed on Feb. 25, 2021, 2 pages. [cited by applicant]
Deictic expressions, Retrieved From: https://web.archive.org/web/20140420003652/http:/www.stibaiecjakarta.ac.id/index.php/78-news/64-deictic-expressions, Retrieved on Apr. 30, 2017, 4 Pages. [cited by applicant]
Devault., et al., “Can I finish? Learning when to respond to incremental interpretation results in interactive dialogue,” In Proceedings of 10th Annual SIGDIAL meeting on Discourse and Dialogue, 2009, 10 Pages. [cited by applicant]
Devault., et al., “Managing ambiguities across utterances in dialogue,” In Proceedings of the 11th Workshop on the Semantics and Pragmatics of Dialogue, 2007, 8 Pages. [cited by applicant]
Dockstader., et al., “Multiple Camera Tracking of Interacting and Occluded Human Motion,” In Proceedings of the IEEE, vol. 89, Issue 10, 2001, 30 Pages. [cited by applicant]
Dylgjeri., et al., “Deixis in Modern Linguistics and Outside,” In Academic Journal of Interdisciplinary Studies, vol. 2, Issue 4, 2013, pp. 87-96. [cited by applicant]
Erdem., et al., “Fusing Inertial Sensor Data in an Extended Kalman Filter for 3D Camera Tracking,” In Proceedings of IEEE Transactions on Image Processing, vol. 24, Issue 2, 2014, 3 Pages. [cited by applicant]
First Office Action and Search Report Issued in Chinese Patent Application No. 201880011946.4, mailed on Mar. 1, 2021, 15 pages. [cited by applicant]
He., et al., “An Inertial and Optical Sensor Fusion Approach for Six Degree-of-Freedom Pose Estimation,” In journal of Sensors, vol. 15, Issue 7, 2015, pp. 16448-16465. [cited by applicant]
Horvitz., et al., “Attention-Sensitive Alerting,” In Proceedings of the Conference on Uncertainty and Artificial intelligence, Jul. 1999, 10 Pages. [cited by applicant]
Jacada Intelligent Assistant, Retrieved From: https://www.jacada.com/products/intelligent-assistan, 2017, 7 Pages. [cited by applicant]
Jeong, et al., “Kalman Filter Based Multiple Objects Detection-Tracking Algorithm Robust to Occlusion”, In Proceedings of the SICE Annual Conference, Sep. 2014 pp. 941-946. [cited by applicant]
Kasai, et al., “Adaptive notification system guaranteeing message reachability”, In Proceedings of International Conference on Pervasive Services, Sep. 2005, pp. 208-217. [cited by applicant]
Kozub, et al., “How to use Calendar with Siri”, Retrieved From: http://www.imore.com/how-to-use-calendar-with-siri, Apr. 12, 2016, 7 Pages. [cited by applicant]
Krause, Andreas, “Near-optimal Sensor Placements: Maximizing Information while Minimizing Communication Cost”, In Technical Report of CMU-CALD-05-IIO, 2005, 19 Pages. [cited by applicant]
Lorenz, Trueba, “Noise robust Voice Activity Detection for multiple speakers”, In Proceedings of International Symposium on Intelligent Signal Processing and Communication Systems, Dec. 2010, 4 Pages. [cited by applicant]
Marbini, et al., “Adaptive Sampling Mechanisms in Sensor Networks”, In Proceedings of London Communications Symposium, 2003, 4 Pages. [cited by applicant]
Meng., et al., “Hierarchical RNN with Static Sentence-Level Attention for Text-Based Speaker Change Detection,” In Journal of Computing Research Repository, 2017, 5 Pages. [cited by applicant]
Nguyen, et al., “An Agent-Based Approach to Dialogue Management in Personal Assistants,” In Proceedings of the 10th international conference on Intelligent user interfaces, 2005, 9 Pages. [cited by applicant]
Notice of Allowance Issued in Chinese Patent Applicant No. 201880011578.3 mailed on Sep. 18, 2021, 4 pages. (English Translation Provided). [cited by applicant]
Notice of Allowance Issued in Chinese Patent Applicant No. 201880011946.4 mailed on Jul. 26, 2021, 5 pages. (English Translation Provided). [cited by applicant]
Notice of Allowance Issued in Chinese Patent Applicant No. 201880012028.3 mailed on Sep. 15, 2021, 4 pages. (English Translation Provided). [cited by applicant]
Notice of Allowance Issued in Chinese Patent Application No. 201880011716.8 mailed on Jan. 26, 2022, 5 pages. (English Translation Provided). [cited by applicant]
Notice of Allowance Issued in Chinese Patent Application No. 201880011967.6 mailed on Sep. 7, 2021, 5 pages. (English Translation Provided). [cited by applicant]
Notice of Allowance Issued in Chinese Patent Application No. 201880011970.8 mailed on Jun. 29, 2021, 4 pages. (English Translation Provided). [cited by applicant]
Notice of Allowance Issued in European Patent Application No. 18706104.9 mailed on Dec. 23, 2021, 7 pages. [cited by applicant]
Notice of Allowance Issued in European Patent Application No. 18706104.9 mailed on Mar. 3, 2022, 3 pages. [cited by applicant]
Notice of Allowance Issued in European Patent Application No. 18706619.6 mailed on Jun. 17, 2020, 7 pages. [cited by applicant]
Notice of Allowance Issued in European Patent Application No. 18706619.6 mailed on Sep. 18, 2020, 7 pages. [cited by applicant]
Notice of Allowance Issued in European Patent Application No. 18706620.4 mailed on Jun. 25, 2020, 7 pages. [cited by applicant]
Notice of Allowance Issued in European Patent Application No. 18706620.4 mailed on Oct. 1, 2020, 2 pages. [cited by applicant]
Notice of Allowance Issued in European Patent Application No. 18707800.1, mailed on Oct. 6, 2020, 7 pages. [cited by applicant]
Notice of Allowance Issued in European Patent Application No. 18708508.9 mailed on Aug. 12, 2021, 2 pages. [cited by applicant]
Notice of Allowance Issued in European Patent Application No. 18708508.9 mailed on May 17, 2021, 7 pages. [cited by applicant]
Notice of allowance mailed on Jun. 5, 2019, in U.S. Appl. No. 15/656,994, 7 pages. [cited by applicant]
Notice of allowance mailed on May 20, 2019, in U.S. Appl. No. 15/636,559, 7 Pages. [cited by applicant]
Paek, et al., “Continuous Listening for Unconstrained Spoken Dialog,” In Proceedings of the Sixth International Conference on Spoken Language Processing, 2000, 4 Pages. [cited by applicant]
Rachuri, et al., “METIS: Exploring Mobile Phone Sensing Offloading for Efficiently Supporting Social Sensing Applications,” In Proceedings of IEEE International Conference on Pervasive Computing and Communications, 2013… [cited by applicant]
Rubinstein, et al., “Analysis and Visualization of Temporal Variations in Video,” Thesis Submitted to the Department of Electrical Engineering and Computer Science in Partial Fulfilment of the Requirements for the Degre… [cited by applicant]
Spors, et al., “A Multi-Sensor Object Localization System,” In Proceedings of the Vision Modeling and Visualization Conference, vol. 1, 2001, pp. 19-26. [cited by applicant]
Vastenburg, et al., “Considerate Home Notification Systems: A User Study of Acceptability of Notifications in a Living Room Laboratory,” In International Journal of Human-Computer Studies, vol. 67, 2009, pp. 814-826. [cited by applicant]
Wu, et al., “Eulerian Video Magnification for Revealing Subtle Changes in the World,” In Journal of ACM Transactions on Graphics, vol. 31, No. 4, Jul. 2012, 9 pages. [cited by applicant]
Xiong, et al., “Human Tracking System Based on PIR Sensor Network and Video,” In Proceedings of Advanced Technologies in Ad Hoc and Sensor Networks, 2014, pp. 13-25. [cited by applicant]
Zhang, et al., “Location Management in Cellular Networks,” In Publication of John Wiley & Sons, 2002, 26 Pages. [cited by applicant]
Eckert, et al., “Resolving Discourse Deictic Anaphora in Dialogues,” In Proceedings of 9th Conference of the European Chapter of the Association for Computational Linguistics, 1999, 8 Pages. [cited by applicant]
First Office Action Received for Chinese Application No. 201980022427.2 mailed on Aug. 17, 2023, 28 pages. (English Translation Provided). [cited by applicant]
Kong, et al., “Selecting Home Appliances with Smart Glass based on Contextual Information”, In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing, Sep. 2016, pp. 97-108. [cited by applicant]
Liang, et al., “Semantic Object Parsing with Graph LSTM”, In Journal of Computing Research Repository, 2016, 16 Pages. [cited by applicant]
Madden, et al., “TAG: a Tiny AGgregation Service for Ad-Hoc Sensor Networks”, In Proceedings 5th Symposium on Operating System Design and Implementation, 2002, pp. 131-146. [cited by applicant]
Marathe, et al., “What drives customization? control or identity?”, In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, May 2011, pp. 781-790. [cited by applicant]
Mesnil, et al., “Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding,” In Journal of IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 23, No. 3, 2013, pp. 530-539. [cited by applicant]
Minotto, et al., “Simultaneous-Speaker Voice Activity Detection and Localization Using Mid-Fusion of SVM and HMMs,” In Proceedings of IEEE Transactions on Multimedia, Jun. 2014, vol. 16, No. 4, pp. 1032-1044. [cited by applicant]
Mrazovac, et al., “Smart Audio/Video Playback Control Based on Presence Detection and User Localization in Home Environment,” In Proceedings of the Second Eastern European Regional Conference on the Engineering of Compu… [cited by applicant]
Notice of Allowance Issued in U.S. Appl. No. 15/657,031, mailed on Mar. 18, 2019,7 Pages. [cited by applicant]
Notice of Allowance Issued in U.S. Appl. No. 15/657,031, mailed on Jun. 26, 2019, 8 pages. [cited by applicant]
Notice of Allowance Issued in U.S. Appl. No. 15/980,631, mailed on Mar. 31, 2021, 6 Pages. [cited by applicant]
Notice of Allowance Issued in U.S. Appl. No. 16/700,308, mailed on Feb. 3, 2021, 8 Pages. [cited by applicant]
Notice of Allowance Mailed Dec. 11, 2019, in U.S. Appl. No. 15/682,407, 11 Pages. [cited by applicant]
Notice of Allowance Mailed Oct. 2, 2019, in U.S. Appl. No. 15/682,425, 8 pages. [cited by applicant]
Notice of allowance mailed on Dec. 3, 2020, in U.S. Appl. No. 15/640,251, 11 Pages. [cited by applicant]
Notice of allowance mailed on Jan. 17, 2019, in U.S. Appl. No. 15/636,559, 10 pages. [cited by applicant]
Notice of Allowance mailed on Jan. 27, 2021, in U.S. Appl. No. 15/936,076, 10 pages. [cited by applicant]
Notice of allowance mailed on Jun. 1, 2021, in U.S. Appl. No. 16/599,426, 7 Pages. [cited by applicant]
Notice of Allowance mailed on Sep. 18, 2020, in U.S. Appl. No. 15/936,076, 10 pages. [cited by applicant]
Office Action Issued in European Patent Application No. 22153942.2 mailed on Jul. 18, 2022, 2 pages. [cited by applicant]
Verger, et al., “Samsung's New Digital Assistant, Bixby, Tries to Push Past Voice Recognition Toward true Al,” The Galaxy S8 smartphone Can See, Listen, and Learn, Retrieved From: http://www.popsci.com/bixby-samsung-ass… [cited by applicant]
Yin, et al., “Object Localization and Tracking Based on Multiple Sensor Fusion in Intelligent Home,” In Proceedings of the 26th Chinese Control and Decision Conference, 2014, pp. 5266-5270. [cited by applicant]
Decision to grant a European patent pursuant to Article 97(1) received in European Application No. 19727161.2, mailed om Jan. 16, 2025, 2 pages. [cited by applicant]
Hearing Notice Issued in Indian Patent Application No. 201947030165, mailed on Jan. 20, 2025, 3 Pages. [cited by applicant]