IP Library › Granted Patent US 12,238,496
Granted Patent B2
US 12,238,496 · App. 18/418,131 · Granted Feb 25, 2025

Method of waking a device using spoken voice commands

Inventors: David Thomas Roach (Plantation, FL); Jean-Marc Jot (Aptos, CA); Jung-Suk Lee (Santa Clara, CA)
Assignee: Magic Leap, Inc.
H04R5/04G02B27/0172G10L15/08H04R3/005H04R3/04H04R5/033G06T19/006
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,238,496
App. No.
18/418,131
Granted
Feb 25, 2025
Kind
B2
Abstract

Disclosed herein are systems and methods for processing speech signals in mixed reality applications. A method may include receiving an audio signal; determining, via first processors, whether the audio signal comprises a voice onset event; in accordance with a determination that the audio signal comprises the voice onset event: waking a second one or more processors; determining, via the second processors, that the audio signal comprises a predetermined trigger signal; in accordance with a determination that the audio signal comprises the predetermined trigger signal: waking third processors; performing, via the third processors, automatic speech recognition based on the audio signal; and in accordance with a determination that the audio signal does not comprise the predetermined trigger signal: forgoing waking the third processors; and in accordance with a determination that the audio signal does not comprise the voice onset event: forgoing waking the second processors.

Claims (101)

1. A system comprising:

a head-wearable device configured to be worn by a user, the head-wearable device comprising:

a first microphone configured to rest at a first height with respect to the user's mouth when the head-wearable device is worn by the user;

a second microphone configured to rest at a second height with respect to the user's mouth when the head-wearable device is worn by the user, the second height different than the first height; and

one or more processors configured to perform a method comprising:

receiving, via the first microphone, a first microphone output based on an audio signal, the audio signal provided by the user;

receiving, via the second microphone, a second microphone output based on the audio signal;

determining, via a first processor of the one or more processors, whether the audio signal comprises a voice onset event;

in accordance with a determination that the audio signal comprises the voice onset event:

waking a second processor of the one or more processors;

determining, via the second processor of the one or more processors, whether the audio signal comprises a predetermined trigger signal;

in accordance with a determination that the audio signal comprises the predetermined trigger signal:

 waking a third processor of the one or more processors;

 performing, via the third processor of the one or more processors, automatic speech recognition based on the audio signal; and

in accordance with a determination that the audio signal does not comprise the predetermined trigger signal:

 forgoing waking the third processor of the one or more processors; and

in accordance with a determination that the audio signal does not comprise the voice onset event:

forgoing waking the second processor of the one or more processors,

wherein said determining whether the audio signal comprises the voice onset event comprises:

determining a first probability of voice activity with respect to the audio signal based on the first microphone output,

determining a second probability of voice activity with respect to the audio signal, wherein the determining the second probability of voice activity comprises performing beamforming based on the first microphone output and the second microphone output, and

determining a combined probability of voice activity based on the first probability of voice activity and further based on the second probability of voice activity.

2. The system of claim 1 , wherein the method further comprises:

in accordance with the determination that the audio signal comprises the predetermined trigger signal:

providing, via the second processor of the one or more processors, an audio stream to the third processor of the one or more processors based on the audio signal;

identifying an endpoint of the audio signal; and

ceasing to provide the audio stream to the third processor of the one or more processors in response to identifying the endpoint.

3. The system of claim 1 , wherein the first processor of the one or more processors comprises one or more of an application-specific integrated circuit or a digital signal processor, configured to determine whether the audio signal comprises the voice onset event.

4. The system of claim 1 , wherein the second processor of the one or more processors comprises one or more of a digital signal processor or an application-specific integrated circuit.

5. The system of claim 1 , wherein:

the head-wearable device comprises the first processor of the one or more processors and the second processor of the one or more processors, and

the system further comprises an auxiliary unit comprising the third processor of the one or more processors, wherein the auxiliary unit is external to the head-wearable device and configured to communicate with the head-wearable device.

6. The system of claim 1 , wherein the predetermined trigger signal comprises a phrase.

7. The system of claim 1 , wherein the method further comprises storing the audio signal in a buffer.

8. The system of claim 1 , wherein the method further comprises:

in accordance with the determination that the audio signal comprises the voice onset event:

performing, via the second processor of the one or more processors, acoustic echo cancellation based on the audio signal.

9. The system of claim 1 , wherein the method further comprises:

in accordance with the determination that the audio signal comprises the voice onset event:

performing, via the second processor of the one or more processors, beamforming based on the audio signal.

10. The system of claim 1 , wherein the method further comprises:

in accordance with the determination that the audio signal comprises the voice onset event:

performing, via the second processor of the one or more processors, noise reduction based on the audio signal.

11. The system of claim 1 , wherein the determining whether the audio signal comprises the voice onset event further comprises determining an amount of voice activity in the audio signal.

12. The system of claim 11 , wherein the determining whether the audio signal comprises the voice onset event further comprises determining whether the amount of voice activity in the audio signal is greater than a voice activity threshold.

13. The system of claim 1 , wherein said determining whether the audio signal comprises the voice onset event comprises determining whether the audio signal comprises a voice of the user.

14. The system of claim 1 , wherein a difference between the first microphone output and the second microphone output indicates whether the audio signal comprises a voice of the user.

15. A method comprising:

receiving, via a first microphone, a first microphone output based on an audio signal, the audio signal provided by a user;

receiving, via a second microphone, a second microphone output based on the audio signal;

determining, via a first processor of the one or more processors, whether the audio signal comprises a voice onset event;

in accordance with a determination that the audio signal comprises the voice onset event:

waking a second processor of the one or more processors;

determining, via the second processor of the one or more processors, whether the audio signal comprises a predetermined trigger signal;

in accordance with a determination that the audio signal comprises the predetermined trigger signal:

waking a third processor of the one or more processors;

performing, via the third processor of the one or more processors, automatic speech recognition based on the audio signal; and

in accordance with a determination that the audio signal does not comprise the predetermined trigger signal:

forgoing waking the third processor of the one or more processors; and

in accordance with a determination that the audio signal does not comprise the voice onset event:

forgoing waking the second processor of the one or more processors, wherein:

a head-wearable device comprises the first microphone and further comprises the second microphone,

the first microphone is configured to rest at a first height with respect to the user's mouth when the head-wearable device is worn by the user,

the second microphone is configured to rest at a second height with respect to the user's mouth when the head-wearable device is worn by the user, the second height different than the first height, and

said determining whether the audio signal comprises the voice onset event comprises:

determining a first probability of voice activity with respect to the audio signal based on the first microphone output,

determining a second probability of voice activity with respect to the audio signal, wherein the determining the second probability of voice activity comprises performing beamforming based on the first microphone output and the second microphone output, and

determining a combined probability of voice activity based on the first probability of voice activity and further based on the second probability of voice activity.

16. The method of claim 15 , wherein the head-wearable device comprises the first processor of the one or more processors and the second processor of the one or more processors, wherein an auxiliary unit comprises the third processor of the one or more processors, and wherein the auxiliary unit is external to the head-wearable device and configured to communicate with the head-wearable device.

17. The method of claim 15 , further comprising:

in accordance with the determination that the audio signal comprises the predetermined trigger signal:

providing, via the second processor of the one or more processors, an audio stream to the third processor of the one or more processors based on the audio signal;

identifying an endpoint of the audio signal; and

ceasing to provide the audio stream to the third processor of the one or more processors in response to identifying the endpoint.

18. A non-transitory computer-readable medium storing instructions, which, when executed by one or more processors, cause the one or more processors to perform a method comprising:

receiving, via a first microphone, a first microphone output based on an audio signal, the audio signal provided by a user;

receiving, via a second microphone, a second microphone output based on the audio signal;

determining, via a first processor of the one or more processors, whether the audio signal comprises a voice onset event;

in accordance with a determination that the audio signal comprises the voice onset event:

waking a second processor of the one or more processors;

determining, via the second processor of the one or more processors, whether the audio signal comprises a predetermined trigger signal;

in accordance with a determination that the audio signal comprises the predetermined trigger signal:

waking a third processor of the one or more processors;

performing, via the third processor of the one or more processors, automatic speech recognition based on the audio signal; and

in accordance with a determination that the audio signal does not comprise the predetermined trigger signal:

forgoing waking the third processor of the one or more processors; and

in accordance with a determination that the audio signal does not comprise the voice onset event:

forgoing waking the second processor of the one or more processors, wherein:

a head-wearable device comprises the first microphone and further comprises the second microphone,

the first microphone is configured to rest at a first height with respect to the user's mouth when the head-wearable device is worn by the user,

the second microphone is configured to rest at a second height with respect to the user's mouth when the head-wearable device is worn by the user, the second height different than the first height, and

said determining whether the audio signal comprises the voice onset event comprises:

determining a first probability of voice activity with respect to the audio signal based on the first microphone output,

determining a second probability of voice activity with respect to the audio signal, wherein the determining the second probability of voice activity comprises performing beamforming based on the first microphone output and the second microphone output, and

determining a combined probability of voice activity based on the first probability of voice activity and further based on the second probability of voice activity.

19. The non-transitory computer-readable medium of claim 18 , wherein the head-wearable device comprises the first processor of the one or more processors and the second processor of the one or more processors, wherein an auxiliary unit comprises the third processor of the one or more processors, and wherein the auxiliary unit is external to the head-wearable device and configured to communicate with the head-wearable device.

20. The non-transitory computer-readable medium of claim 18 , the method further comprising:

in accordance with the determination that the audio signal comprises the predetermined trigger signal:

providing, via the second processor of the one or more processors, an audio stream to the third processor of the one or more processors based on the audio signal;

identifying an endpoint of the audio signal; and

ceasing to provide the audio stream to the third processor of the one or more processors in response to identifying the endpoint.

Assignments (3)
SECURITY INTEREST Recorded Oct 31, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073422/0549 →
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073388/0027 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2025
From: ROACH, DAVID THOMAS; JOT, JEAN-MARC; LEE, JUNG-SUK
To: MAGIC LEAP, INC.
Reel/Frame 069824/0760 →
Continuity (4)
Continuation 17214446 · Mar 26, 2021
Provisional Application 63033451 · Jun 2, 2020
Provisional Application 63001116 · Mar 27, 2020
Related Publication 20240163612A1 · May 16, 2024
References Cited (229)
US 4158750A · Sakoe · 1979 [cited by applicant]
US 4852988A · Velez · 1989 [cited by applicant]
US 6433760B1 · Vaissie · 2002 [cited by applicant]
US 6491391B1 · Blum et al. · 2002 [cited by applicant]
US 6496799B1 · Pickering · 2002 [cited by applicant]
US 6847336B1 · Lemelson · 2005 [cited by applicant]
US 6943754B2 · Aughey · 2005 [cited by applicant]
US 6977776B2 · Volkenandt et al. · 2005 [cited by applicant]
US 7346654B1 · Weiss · 2008 [cited by applicant]
US 7347551B2 · Fergason et al. · 2008 [cited by applicant]
US 7488294B2 · Torch · 2009 [cited by applicant]
US 7587319B2 · Catchpole · 2009 [cited by applicant]
US 7979277B2 · Larri et al. · 2011 [cited by applicant]
US 8154588B2 · Burns · 2012 [cited by applicant]
US 8235529B1 · Raffle · 2012 [cited by applicant]
US 8611015B2 · Wheeler · 2013 [cited by applicant]
US 8638498B2 · Bohn et al. · 2014 [cited by applicant]
US 8696113B2 · Lewis · 2014 [cited by applicant]
US 8929589B2 · Publicover et al. · 2015 [cited by applicant]
US 9010929B2 · Lewis · 2015 [cited by applicant]
US 9274338B2 · Robbins et al. · 2016 [cited by applicant]
US 9292973B2 · Bar-zeev et al. · 2016 [cited by applicant]
US 9294860B1 · Carlson · 2016 [cited by applicant]
US 9323325B2 · Perez et al. · 2016 [cited by applicant]
US 9715875B2 · Piernot · 2017 [cited by applicant]
US 9720505B2 · Gribetz et al. · 2017 [cited by applicant]
US 10013053B2 · Cederlund et al. · 2018 [cited by applicant]
US 10025379B2 · Drake et al. · 2018 [cited by applicant]
US 10062377B2 · Larri et al. · 2018 [cited by applicant]
US 10134425B1 · Johnson, Jr. · 2018 [cited by applicant]
US 10289205B1 · Sumter · 2019 [cited by applicant]
US 10839789B2 · Larri et al. · 2020 [cited by applicant]
US 10971140B2 · Catchpole · 2021 [cited by applicant]
US 11328740B2 · Lee et al. · 2022 [cited by applicant]
US 11587563B2 · Sheeder et al. · 2023 [cited by applicant]
US 11790935B2 · Lee et al. · 2023 [cited by applicant]
US 11854550B2 · Sheeder et al. · 2023 [cited by applicant]
US 11854566B2 · Leider · 2023 [cited by applicant]
US 11917384B2 · Roach · 2024 [cited by applicant]
US 12094489B2 · Lee · 2024 [cited by applicant]
US 20010055985A1 · Matt et al. · 2001 [cited by applicant]
US 20030030597A1 · Geist · 2003 [cited by applicant]
US 20050033571A1 · Huang · 2005 [cited by applicant]
US 20050069852A1 · Janakiraman et al. · 2005 [cited by applicant]
US 20060023158A1 · Howell et al. · 2006 [cited by applicant]
US 20060072767A1 · Zhang et al. · 2006 [cited by applicant]
US 20060098827A1 · Paddock et al. · 2006 [cited by applicant]
US 20060178876A1 · Sato et al. · 2006 [cited by applicant]
US 20070225982A1 · Washio · 2007 [cited by applicant]
US 20080124690A1 · Redlich · 2008 [cited by applicant]
US 20080201138A1 · Visser et al. · 2008 [cited by applicant]
US 20090180626A1 · Nakano · 2009 [cited by applicant]
US 20100245585A1 · Fisher et al. · 2010 [cited by applicant]
US 20100323652A1 · Visser et al. · 2010 [cited by applicant]
US 20110211056A1 · Publicover et al. · 2011 [cited by applicant]
US 20110213664A1 · Osterhout · 2011 [cited by applicant]
US 20110238407A1 · Kent · 2011 [cited by applicant]
US 20110288860A1 · Schevciw et al. · 2011 [cited by applicant]
US 20120021806A1 · Maltz · 2012 [cited by applicant]
US 20120130713A1 · Shin et al. · 2012 [cited by applicant]
US 20120209601A1 · Jing · 2012 [cited by applicant]
US 20130077147A1 · Efimov · 2013 [cited by applicant]
US 20130204607A1 · Baker, IV · 2013 [cited by applicant]
US 20130339028A1 · Rosner et al. · 2013 [cited by applicant]
US 20140016793A1 · Gardner · 2014 [cited by applicant]
US 20140194702A1 · Tran · 2014 [cited by applicant]
US 20140195918A1 · Friedlander · 2014 [cited by applicant]
US 20140200887A1 · Nakadai et al. · 2014 [cited by applicant]
US 20140222430A1 · Rao · 2014 [cited by applicant]
US 20140270202A1 · Ivanov et al. · 2014 [cited by applicant]
US 20140270244A1 · Fan · 2014 [cited by applicant]
US 20140337023A1 · Mcculloch et al. · 2014 [cited by applicant]
US 20140379336A1 · Bhatnagar · 2014 [cited by applicant]
US 20150006181A1 · Fan et al. · 2015 [cited by applicant]
US 20150168731A1 · Robbins · 2015 [cited by applicant]
US 20150310857A1 · Habets et al. · 2015 [cited by applicant]
US 20150348572A1 · Thornburg et al. · 2015 [cited by applicant]
US 20160019910A1 · Faubel et al. · 2016 [cited by applicant]
US 20160066113A1 · Elkhatib et al. · 2016 [cited by applicant]
US 20160112817A1 · Fan et al. · 2016 [cited by applicant]
US 20160142830A1 · Hu · 2016 [cited by applicant]
US 20160165340A1 · Benattar · 2016 [cited by applicant]
US 20160180837A1 · Gustavsson · 2016 [cited by applicant]
US 20160216130A1 · Abramson et al. · 2016 [cited by applicant]
US 20160217781A1 · Zhong et al. · 2016 [cited by applicant]
US 20160284350A1 · Yun et al. · 2016 [cited by applicant]
US 20160358598A1 · Williams et al. · 2016 [cited by applicant]
US 20160379629A1 · Hofer et al. · 2016 [cited by applicant]
US 20160379632A1 · Hoffmeister et al. · 2016 [cited by applicant]
US 20160379638A1 · Basye et al. · 2016 [cited by applicant]
US 20170078819A1 · Habets · 2017 [cited by applicant]
US 20170091169A1 · Bellegarda · 2017 [cited by applicant]
US 20170092276A1 · Sun et al. · 2017 [cited by applicant]
US 20170110116A1 · Tadpatrikar et al. · 2017 [cited by applicant]
US 20170148429A1 · Hayakawa · 2017 [cited by applicant]
US 20170270919A1 · Parthasarathi et al. · 2017 [cited by applicant]
US 20170280239A1 · Sekiya · 2017 [cited by applicant]
US 20170316780A1 · Lovitt · 2017 [cited by applicant]
US 20170330555A1 · Kawano · 2017 [cited by applicant]
US 20170332187A1 · Lin · 2017 [cited by applicant]
US 20180011534A1 · Poulos et al. · 2018 [cited by applicant]
US 20180053284A1 · Rodriguez et al. · 2018 [cited by applicant]
US 20180077095A1 · Deyle et al. · 2018 [cited by applicant]
US 20180129469A1 · Vennström et al. · 2018 [cited by applicant]
US 20180227665A1 · Elko et al. · 2018 [cited by applicant]
US 20180316939A1 · Todd · 2018 [cited by applicant]
US 20180336902A1 · Cartwright et al. · 2018 [cited by applicant]
US 20180349946A1 · Nguyen · 2018 [cited by applicant]
US 20180358021A1 · Mistica et al. · 2018 [cited by applicant]
US 20180366114A1 · Anbazhagan et al. · 2018 [cited by applicant]
US 20190129944A1 · Kawano · 2019 [cited by applicant]
US 20190362741A1 · Li et al. · 2019 [cited by applicant]
US 20190373362A1 · Ansai et al. · 2019 [cited by applicant]
US 20190392641A1 · Taylor · 2019 [cited by applicant]
US 20200027455A1 · Sugiyama et al. · 2020 [cited by applicant]
US 20200064921A1 · Kang et al. · 2020 [cited by applicant]
US 20200194028A1 · Lipman · 2020 [cited by applicant]
US 20200213729A1 · Soto · 2020 [cited by applicant]
US 20200279552A1 · Piersol et al. · 2020 [cited by applicant]
US 20200286465A1 · Wang et al. · 2020 [cited by applicant]
US 20200296521A1 · Wexler et al. · 2020 [cited by applicant]
US 20200335128A1 · Sheeder et al. · 2020 [cited by applicant]
US 20210056966A1 · Bilac et al. · 2021 [cited by applicant]
US 20210125609A1 · Dusan et al. · 2021 [cited by applicant]
US 20210192413A1 · Shirazipour · 2021 [cited by applicant]
US 20230386461A1 · Leider · 2023 [cited by applicant]
US 20230410835A1 · Lee · 2023 [cited by applicant]
US 20240087565A1 · Sheeder · 2024 [cited by applicant]
US 20240087587A1 · Leider · 2024 [cited by applicant]
CA 2316473A1 · 2001 [cited by applicant]
CA 2362895A1 · 2002 [cited by applicant]
CA 2388766A1 · 2003 [cited by applicant]
CN 105529033A · 2016 [cited by applicant]
EP 2950307A1 · 2015 [cited by applicant]
EP 3211918A1 · 2017 [cited by applicant]
JP S52144205A · 1977 [cited by applicant]
JP 2000148184A · 2000 [cited by applicant]
JP 2002135173A · 2002 [cited by applicant]
JP 2005196134A · 2005 [cited by applicant]
JP 2014137405A · 2014 [cited by applicant]
JP 2014178339A · 2014 [cited by applicant]
JP 2016004270A · 2016 [cited by applicant]
JP 2017211596A · 2017 [cited by applicant]
JP 2018523156A · 2018 [cited by applicant]
JP 2018179954A · 2018 [cited by applicant]
WO 2014113891A1 · 2014 [cited by applicant]
WO 2014159581A1 · 2014 [cited by applicant]
WO 2015169618A1 · 2015 [cited by applicant]
WO 2016063587A1 · 2016 [cited by applicant]
WO 2016151956A1 · 2016 [cited by applicant]
WO 2016153712A1 · 2016 [cited by applicant]
WO 2017003903A1 · 2017 [cited by applicant]
WO 2017017591A1 · 2017 [cited by applicant]
WO 2017191711A1 · 2017 [cited by applicant]
WO 2019224292A1 · 2019 [cited by applicant]
WO 2020180719A1 · 2020 [cited by applicant]
WO 2020214844A1 · 2020 [cited by applicant]
WO 2022072752A1 · 2022 [cited by applicant]
WO 2023064875A1 · 2023 [cited by applicant]
Backstrom, T. (Oct. 2015). “Voice Activity Detection Speech Processing,” Aalto University, vol. 58, No. 10; Publication [online], retrieved Apr. 19, 2020, retrieved from the Internet: URL: https://mycourses.aalto.fi/plu… [cited by applicant]
Bilac, M. et al. (Nov. 15, 2017). Gaze and Filled Pause Detection for Smooth Human-Robot Conversations. www.angelicalim.com , retrieved on Jun. 17, 2020, Retrieved from the internet URL: http://www.angelicalim.com/paper… [cited by applicant]
Chinese Office Action dated Dec. 21, 2023, for CN Application No. 201980050714.4, with English translation, eighteen pages. [cited by applicant]
European Office Action dated Jun. 1, 2023, for EP Application No. 19822754.8, six pages. [cited by applicant]
European Search Report dated Nov. 12, 2021, for EP Application No. 19822754.8, ten pages. [cited by applicant]
European Search Report dated Nov. 21, 2022, for EP Application No. 20791183.5 nine pages. [cited by applicant]
European Search Report dated Oct. 6, 2022, for EP Application No. 20766540.7 nine pages. [cited by applicant]
Final Office Action mailed Apr. 10, 2023, for U.S. Appl. No. 16/850,965, filed Apr. 16, 2020, sixteen pages. [cited by applicant]
Final Office Action mailed Apr. 15, 2022, for U.S. Appl. No. 16/850,965, filed Apr. 16, 2020, fourteen pages. [cited by applicant]
Final Office Action mailed Aug. 4, 2023, for U.S. Appl. No. 17/254,832, filed Dec. 21, 2020, seventeen pages. [cited by applicant]
Final Office Action mailed Aug. 5, 2022, for U.S. Appl. No. 16/805,337, filed Feb. 28, 2020, eighteen pages. [cited by applicant]
Final Office Action mailed Jan. 11, 2023, for U.S. Appl. No. 17/214,446, filed Mar. 26, 2021, sixteen pages. [cited by applicant]
Final Office Action mailed Jan. 23, 2024, for U.S. Appl. No. 16/850,965, filed Apr. 16, 2020, fifteen pages. [cited by applicant]
Final Office Action mailed Oct. 6, 2021, for U.S. Appl. No. 16/805,337, filed Feb. 28, 2020, fourteen pages. [cited by applicant]
Final Office Action mailed Sep. 7, 2023, for U.S. Appl. No. 17/214,446, filed Mar. 26, 2021, nineteen pages. [cited by applicant]
Harma, A. et al. (Jun. 2004). “Augmented Reality Audio for Mobile and Wearable Appliances,” J. Audio Eng. Soc., vol. 52, No. 6, retrieved on Aug. 20, 2019, Retrieved from the Internet: URL:https://pdfs.semanticscholar.o… [cited by applicant]
International Preliminary Report and Patentability mailed Dec. 22, 2020, for PCT Application No. PCT/US2019/038546, thirteen pages. [cited by applicant]
International Preliminary Report and Written Opinion mailed Apr. 13, 2023, for PCT Application No. PCT/US2021/53046, filed Sep. 30, 2021, nine pages. [cited by applicant]
International Preliminary Report and Written Opinion mailed Oct. 28, 2021, for PCT Application No. PCT/US2020/028570, filed Apr. 16, 2020, seventeen pages. [cited by applicant]
International Preliminary Report and Written Opinion mailed Sep. 16, 2021, for PCT Application No. PCT/US20/20469, filed Feb. 28, 2020, nine pages. [cited by applicant]
International Search Report and Written Opinion mailed Jan. 17, 2023, for PCT Application No. PCT/US22/78073, thirteen pages. [cited by applicant]
International Search Report and Written Opinion mailed Jan. 24, 2022, for PCT Application No. PCT/US2021/53046, filed Sep. 30, 2021, fifteen pages. [cited by applicant]
International Search Report and Written Opinion mailed Jul. 2, 2020, for PCT Application No. PCT/US2020/028570, filed Apr. 16, 2020, nineteen pages. [cited by applicant]
International Search Report and Written Opinion mailed May 18, 2020, for PCT Application No. PCT/US20/20469, filed Feb. 28, 2020, twenty pages. [cited by applicant]
International Search Report and Written Opinion mailed Sep. 17, 2019, for PCT Application No. PCT/ US2019/038546, sixteen pages. [cited by applicant]
Jacob, R. “Eye Tracking in Advanced Interface Design”, Virtual Environments and Advanced Interface Design, Oxford University Press, Inc. (Jun. 1995). [cited by applicant]
Japanese Notice of Allowance mailed Dec. 15, 2023, for JP Application No. 2020-571488, with English translation, eight pages. [cited by applicant]
Japanese Office Action dated Jun. 2, 2023, for JP Application No. 2020 571488, with English translation, nine pages. [cited by applicant]
Japanese Office Action mailed Jan. 30, 2024, for JP Application No. 2021-551538, with English translation, sixteen pages. [cited by applicant]
Kitayama, K. et al. (Sep. 30, 2003). “Speech Starter: Noise-Robust Endpoint Detection by Using Filled Pauses.” Eurospeech 2003, retrieved on Jun. 17, 2020, retrieved from the internet URL: http://clteseerx.ist.psu.edu/v… [cited by applicant]
Liu, Baiyang, et al.: (Sep. 6, 2015). “Accurate Endpointing with Expected Pause Duration,” Interspeech 2015, pp. 2912-2916, retrieved from: a href=“https://scholar.google.com/scholar?q=BAIYANG”target=“_blank”https://sch… [cited by applicant]
Non-Final Office Action mailed Apr. 12, 2023, for U.S. Appl. No. 17/214,446, filed Mar. 26, 2021, seventeen pages. [cited by applicant]
Non-Final Office Action mailed Apr. 13, 2023, for U.S. Appl. No. 17/714,708, filed Apr. 6, 2022, sixteen pages. [cited by applicant]
Non-Final Office Action mailed Apr. 27, 2023, for U.S. Appl. No. 17/254,832, filed Dec. 21, 2020, fourteen pages. [cited by applicant]
Non-Final Office Action mailed Aug. 10, 2022, for U.S. Appl. No. 17/214,446, filed Mar. 26, 2021, fifteen pages. [cited by applicant]
Non-Final Office Action mailed Jun. 23, 2023, for U.S. Appl. No. 18/148,221, filed Dec. 29, 2022, thirteen pages. [cited by applicant]
Non-Final Office Action mailed Jun. 24, 2021, for U.S. Appl. No. 16/805,337, filed Feb. 28, 2020, fourteen pages. [cited by applicant]
Non-Final Office Action mailed Mar. 17, 2022, for U.S. Appl. No. 16/805,337, filed Feb. 28, 2020, sixteen pages. [cited by applicant]
Non-Final Office Action mailed Nov. 17, 2021, for U.S. Appl. No. 16/987,267, filed Aug. 6, 2020, twenty-one pages. [cited by applicant]
Non-Final Office Action mailed Oct. 4, 2021, for U.S. Appl. No. 16/850,965, filed Apr. 16, 2020, twelve pages. [cited by applicant]
Non-Final Office Action mailed Sep. 15, 2023, for U.S. Appl. No. 16/850,965, filed Apr. 16, 2020, fourteen pages. [cited by applicant]
Non-Final Office Action mailed Sep. 29, 2022, for U.S. Appl. No. 16/850,965, filed Apr. 16, 2020, fifteen pages. [cited by applicant]
Notice of Allowance mailed Dec. 15, 2023, for U.S. Appl. No. 17/214,446, filed Mar. 26, 2021, seven pages. [cited by applicant]
Notice of Allowance mailed Jul. 31, 2023, for U.S. Appl. No. 17/714,708, filed Apr. 6, 2022, eight pages. [cited by applicant]
Notice of Allowance mailed Mar. 3, 2022, for U.S. Appl. No. 16/987,267, filed Aug. 6, 2020, nine pages. [cited by applicant]
Notice of Allowance mailed Nov. 30, 2022, for U.S. Appl. No. 16/805,337, filed Feb. 28, 2020, six pages. [cited by applicant]
Notice of Allowance mailed Oct. 12, 2023, for U.S. Appl. No. 18/148,221, filed Dec. 29, 2022, five pages. [cited by applicant]
Notice of Allowance mailed Oct. 17, 2023, for U.S. Appl. No. 17/254,832, filed Dec. 21, 2020, sixteen pages. [cited by applicant]
Rolland, J. et al., “High-resolution inset head-mounted display”, Optical Society of America, vol. 37, No. 19, Applied Optics, (Jul. 1, 1998). [cited by applicant]
Shannon, Matt et al. (Aug. 20-24, 2017). “Improved End-of-Query Detection for Streaming Speech Recognition”, Interspeech 2017, Stockholm, Sweden, pp. 1909-1913. [cited by applicant]
Tanriverdi, V. et al. (Apr. 2000). “Interacting With Eye Movements in Virtual Environments,” Department of Electrical Engineering and Computer Science, Tufts University, Medford, MA 02155, USA, Proceedings of the SIGCHI… [cited by applicant]
Tonges, R. (Dec. 2015). “An augmented Acoustics Demonstrator with Realtime stereo up-mixing and Binaural Auralization,” Technische University Berlin, Audio Communication Group, retrieved on Aug. 22, 2019, Retrieved from… [cited by applicant]
Yoshida, A. et al., “Design and Applications of a High Resolution Insert Head Mounted Display”, (Jun. 1994). [cited by applicant]
European Office Action dated Dec. 12, 2023, for EP Application No. 20766540.7, four pages. [cited by applicant]
International Preliminary Report and Written Opinion mailed Apr. 25, 2024, for PCT Application No. PCT/US2022/078063, seven pages. [cited by applicant]
International Preliminary Report on Patentability and Written Opinion mailed May 2, 2024, for PCT Application No. PCT/US2022/078298, twelve pages. [cited by applicant]
International Search Report and Written Opinion mailed Jan. 11, 2023, for PCT Application No. PCT/US2022/078298, seventeen pages. [cited by applicant]
International Search Report and Written Opinion mailed Jan. 25, 2023, for PCT Application No. PCT/US2022/078063, nineteen pages. [cited by applicant]
Japanese Office Action mailed May 2, 2024, for JP Application No. 2021-562002, with English translation, sixteen pages. [cited by applicant]
Notice of Allowance mailed Jun. 5, 2024, for U.S. Appl. No. 18/459,342, filed Aug. 31, 2023, eight pages. [cited by applicant]
European Notice of Allowance dated Jul. 15, 2024, for EP Application No. 20791183.5 nine pages. [cited by applicant]
Final Office Action mailed Jul. 17, 2024, for U.S. Appl. No. 16/850,965, filed Apr. 16, 2020, seventeen pages. [cited by applicant]
Japanese Final Office Action mailed May 31, 2024, for JP Application No. 2021-551538, with English translation, eighteen pages. [cited by applicant]
International Preliminary Report on Patentability and Written Opinion mailed Apr. 25, 2024, for PCT Application No. PCT/US2022/078073, seven pages. [cited by applicant]
Non-Final Office Action mailed Mar. 27, 2024, for U.S. Appl. No. 16/850,965, filed Apr. 16, 2020, sixteen pages. [cited by applicant]
Japanese Notice of Allowance mailed Aug. 21, 2024, for JP Application No. 2021-562002, with English translation, 6 pages. [cited by applicant]
Japanese Notice of Allowance mailed Oct. 28, 2024, for JP Application No. 2021-551538 with English translation, 6 pages. [cited by applicant]
Non-Final Office Action mailed Aug. 26, 2024, for U.S. Appl. No. 18/506,866, filed Nov. 10, 2023, twelve pages. [cited by applicant]
Non-Final Office Action mailed Nov. 6, 2024, for U.S. Appl. No. 16/850,965, filed Apr. 16, 2020, twenty pages. [cited by applicant]
Chinese Notice of Allowance dated Nov. 7, 2024, for CN Application No. 201980050714.4, with English translation, 6 pages. [cited by applicant]
Cited By (5)
US 12,327,573 US 12,347,448 US 12,417,766 US 12,688,845 US 12,696,045