IP Library Granted Patent US 12,309,577
Granted Patent B2
US 12,309,577 · App. 18/180,746 · Granted May 20, 2025

Environment acoustics persistence

Inventors: Remi Samuel Audfray (San Francisco, CA); Mark Brandon Hertensteiner (San Jose, CA); Samuel Charles Dicker (San Francisco, CA); Blaine Ivin Wood (Eagle Mountain, UT); Michael Z. Land (Mill Valley, CA); Jean-Marc Jot (Aptos, CA)
Assignee: Magic Leap, Inc.
H04S7/306H04R5/04H04R2430/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,309,577
App. No.
18/180,746
Granted
May 20, 2025
Kind
B2
Abstract

Disclosed herein are systems and methods for storing, organizing, and maintaining acoustic data for mixed reality systems. A system may include one or more sensors of a head-wearable device, a speaker of the head-wearable device, and one or more processors. A method performed by the one or more processors may include receiving a request to present an audio signal. An environment may be identified via the one or more sensors of the head-wearable device. One or more audio model components associated with the environment may be retrieved. A first audio model may be generated based on the audio model components. A second audio model may be generated based on the first audio model. A modified audio signal may be determined based on the second audio model and based on the request to present an audio signal. The modified audio signal may be presented via the speaker of the head-wearable device.

Claims (52)

1. A system comprising:

one or more sensors of a head-wearable device;

a speaker of the head-wearable device;

one or more processors configured to perform a method comprising:

determining, via the one or more sensors of the head-wearable device, a location of the head-wearable device at a first time;

determining whether the location of the head-wearable device is within a threshold distance from a location associated with an audio model, wherein the audio model was updated at a second time before the first time;

in accordance with a determination that the location of the head-wearable device is not within the threshold distance and further in accordance with a determination that a difference between the first time and the second time is not less than a threshold amount of time:

determining, based on audio data, one or more acoustic characteristics of an environment of the head-wearable device, wherein the audio data is received via the one or more sensors of the head-wearable device;

generating an updated audio model based on the one or more acoustic characteristics of the environment; and

presenting, via the speaker of the head-wearable device, a first audio signal based on the updated audio model; and

in accordance with one or more of (1) a determination that the location of the head-wearable device is within the threshold distance and (2) a determination that the difference between the first time and the second time is less than the threshold amount of time:

presenting, via the speaker of the head-wearable device, a second audio signal based on the audio model.

2. The system of claim 1 , wherein the method further comprises receiving the audio model at the head-wearable device via an audio service.

3. The system of claim 1 , wherein:

the method further comprises generating, via an audio service, a notification associated with the updated audio model.

4. The system of claim 1 , wherein the generating the updated audio model comprises generating the updated audio model based on a previous audio model.

5. The system of claim 1 , wherein the one or more acoustic characteristics of the environment comprises one or more dimensions of the environment.

6. The system of claim 1 , wherein the one or more acoustic characteristics of the environment comprises a reverberation time.

7. The system of claim 1 , wherein the one or more acoustic characteristics of the environment comprises a reverberation gain.

8. The system of claim 1 , wherein the one or more acoustic characteristics of the environment comprises a transmission loss coefficient.

9. The system of claim 1 , wherein the one or more acoustic characteristics of the environment comprises an absorption coefficient.

10. The system of claim 1 , wherein the method further comprises:

rendering the first audio signal via the updated audio model, and

rendering the second audio signal via the audio model.

11. The system of claim 1 , wherein the updated audio model is associated with a second location, different from the location associated with the audio model.

12. A method comprising:

determining, via one or more sensors of a head-wearable device, a location of the head-wearable device at a first time;

determining whether the location of the head-wearable device is within a threshold distance from a location associated with an audio model, wherein the audio model was updated at a second time before the first time;

in accordance with a determination that the location of the head-wearable device is not within the threshold distance and further in accordance with a determination that a difference between the first time and the second time is not less than a threshold amount of time:

determining, based on audio data, one or more acoustic characteristics of an environment of the head-wearable device, wherein the audio data is received via the one or more sensors of the head-wearable device;

generating an updated audio model based on the one or more acoustic characteristics of the environment; and

presenting, via a speaker of the head-wearable device, a first audio signal based on the updated audio model; and

in accordance with one or more of (1) a determination that the location of the head-wearable device is within the threshold distance and (2) a determination that the difference between the first time and the second time is less than the threshold amount of time:

presenting, via the speaker of the head-wearable device, a second audio signal based on the audio model.

13. The method of claim 12 , further comprising receiving the audio model at the head-wearable device via an audio service.

14. The method of claim 12 , further comprising generating, via an audio service, a notification associated with the updated audio model.

15. The method of claim 12 , wherein the generating the updated audio model comprises generating the updated audio model based on a previous audio model.

16. The method of claim 12 , wherein the one or more acoustic characteristics of the environment comprises one or more dimensions of the environment.

17. The method of claim 12 , wherein the one or more acoustic characteristics of the environment comprises a reverberation time.

18. The method of claim 12 , wherein the one or more acoustic characteristics of the environment comprises a reverberation gain.

19. The method of claim 12 , wherein the one or more acoustic characteristics of the environment comprises a transmission loss coefficient.

20. The method of claim 12 , wherein the one or more acoustic characteristics of the environment comprises an absorption coefficient.

21. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising:

determining, via one or more sensors of a head-wearable device, a location of the head-wearable device at a first time;

determining whether the location of the head-wearable device is within a threshold distance from a location associated with an audio model, wherein the audio model was updated at a second time before the first time;

in accordance with a determination that the location of the head-wearable device is not within the threshold distance and further in accordance with a determination that a difference between the first time and the second time is not less than a threshold amount of time:

determining, based on audio data, one or more acoustic characteristics of an environment of the head-wearable device, wherein the audio data is received via the one or more sensors of the head-wearable device;

generating an updated audio model based on the one or more acoustic characteristics of the environment; and

presenting, via a speaker of the head-wearable device, a first audio signal based on the updated audio model; and

in accordance with one or more of (1) a determination that the location of the head-wearable device is within the threshold distance and (2) a determination that the difference between the first time and the second time is less than the threshold amount of time:

presenting, via the speaker of the head-wearable device, a second audio signal based on the audio model.

22. The non-transitory computer-readable medium of claim 21 , wherein the method further comprises receiving the audio model at the head-wearable device via an audio service.

Assignments (4)
SECURITY INTEREST Recorded Oct 31, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073439/0168 →
SECURITY INTEREST Recorded Oct 15, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073109/0476 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 28, 2025
From: AUDFRAY, REMI SAMUEL; HERTENSTEINER, MARK BRANDON; DICKER, SAMUEL CHARLES; WOOD, BLAINE IVIN; LAND, MICHAEL Z.; JOT, JEAN-MARC
To: MAGIC LEAP, INC.
Reel/Frame 070670/0365 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 3, 2023
From: AUDFRAY, REMI SAMUEL; HERTENSTEINER, MARK BRANDON; DICKER, SAMUEL CHARLES; WOOD, BLAINE IVIN; LAND, MICHAEL Z.; JOT, JEAN-MARC
To: MAGIC LEAP, INC.
Reel/Frame 064487/0318 →
Continuity (3)
Continuation 17112969 · Dec 4, 2020
Provisional Application 62944956 · Dec 6, 2019
Related Publication 20230239651A1 · Jul 27, 2023
References Cited (67)
US 4852988A · Velez · 1989 [cited by applicant]
US 6433760B1 · Vaissie · 2002 [cited by applicant]
US 6491391B1 · Blum et al. · 2002 [cited by applicant]
US 6847336B1 · Lemelson · 2005 [cited by applicant]
US 6943754B2 · Aughey · 2005 [cited by applicant]
US 6977776B2 · Volkenandt et al. · 2005 [cited by applicant]
US 7347551B2 · Fergason et al. · 2008 [cited by applicant]
US 7488294B2 · Torch · 2009 [cited by applicant]
US 8235529B1 · Raffle · 2012 [cited by applicant]
US 8611015B2 · Wheeler · 2013 [cited by applicant]
US 8638498B2 · Bohn et al. · 2014 [cited by applicant]
US 8696113B2 · Lewis · 2014 [cited by applicant]
US 8929589B2 · Publicover et al. · 2015 [cited by applicant]
US 9010929B2 · Lewis · 2015 [cited by applicant]
US 9274338B2 · Robbins et al. · 2016 [cited by applicant]
US 9292973B2 · Bar-zeev et al. · 2016 [cited by applicant]
US 9323325B2 · Perez et al. · 2016 [cited by applicant]
US 9720505B2 · Gribetz et al. · 2017 [cited by applicant]
US 10013053B2 · Cederlund et al. · 2018 [cited by applicant]
US 10025379B2 · Drake et al. · 2018 [cited by applicant]
US 10440498B1 · Amengual Garí · 2019 [cited by examiner]
US 10972715B1 · Hall et al. · 2021 [cited by applicant]
US 11032664B2 · Walker · 2021 [cited by examiner]
US 11627430B2 · Audfray et al. · 2023 [cited by applicant]
US 20010024504A1 · Jot · 2001 [cited by examiner]
US 20030007648A1 · Currell · 2003 [cited by applicant]
US 20030030597A1 · Geist · 2003 [cited by applicant]
US 20060023158A1 · Howell et al. · 2006 [cited by applicant]
US 20110211056A1 · Publicover et al. · 2011 [cited by applicant]
US 20110213664A1 · Osterhout · 2011 [cited by applicant]
US 20120021806A1 · Maltz · 2012 [cited by applicant]
US 20120093320A1 · Flaks et al. · 2012 [cited by applicant]
US 20120273295A1 · Kolaini et al. · 2012 [cited by applicant]
US 20130041648A1 · Osman · 2013 [cited by applicant]
US 20130077147A1 · Efimov · 2013 [cited by applicant]
US 20130236040A1 · Crawford · 2013 [cited by applicant]
US 20140195918A1 · Friedlander · 2014 [cited by applicant]
US 20150002507A1 · Ambrus et al. · 2015 [cited by applicant]
US 20150168731A1 · Robbins · 2015 [cited by applicant]
US 20150373477A1 · Norris et al. · 2015 [cited by applicant]
US 20170078818A1 · Habets et al. · 2017 [cited by applicant]
US 20170084286A1 · Kim et al. · 2017 [cited by applicant]
US 20180020312A1 · Visser et al. · 2018 [cited by applicant]
US 20180077514A1 · Lee · 2018 [cited by examiner]
US 20190116448A1 · Schmidt · 2019 [cited by applicant]
US 20190166435A1 · Crow et al. · 2019 [cited by applicant]
US 20190261116A1 · Norris · 2019 [cited by examiner]
US 20200059502A1 · Iyer et al. · 2020 [cited by applicant]
US 20200304935A1 · Peters · 2020 [cited by examiner]
US 20210176588A1 · Audfray et al. · 2021 [cited by applicant]
CA 2316473A1 · 2001 [cited by applicant]
CA 2362895A1 · 2002 [cited by applicant]
CA 2388766A1 · 2003 [cited by applicant]
WO 2017205637A1 · 2017 [cited by applicant]
Japanese Notice of Allowance dated Mar. 1, 2023, for JP Application No. 2022-533619, with English translation, six pages. [cited by applicant]
European Search Report dated Apr. 21, 2023, for EP Application No. 20897073.1, eleven pages. [cited by applicant]
Final Office Action mailed Jan. 12, 2022, for U.S. Appl. No. 17/112,969, filed Dec. 4, 2020, seventeen pages. [cited by applicant]
International Preliminary Report and Written Opinion mailed Jun. 16, 2022, for PCT Application No. PCT/US2020/063498, filed Dec. 4, 2020, nine pages. [cited by applicant]
International Search Report and Written Opinion mailed Mar. 9, 2021, for PCT Application No. PCT/US20/63498, filed Dec. 4, 2020, fifteen pages. [cited by applicant]
Jacob, R. “Eye Tracking in Advanced Interface Design”, Virtual Environments and Advanced Interface Design, Oxford University Press, Inc. (Jun. 1995). [cited by applicant]
Non-Final Office Action mailed Jul. 14, 2021, for U.S. Appl. No. 17/112,969, filed Dec. 4, 2020, thirteen pages. [cited by applicant]
Non-Final Office Action mailed Jul. 19, 2022, for U.S. Appl. No. 17/112,969, filed Dec. 4, 2020, eighteen pages. [cited by applicant]
Notice of Allowance mailed Feb. 8, 2023, for U.S. Appl. No. 17/112,969, filed Dec. 4, 2020, ten pages. [cited by applicant]
Rolland, J. et al., “High-resolution inset head-mounted display”, Optical Society of America, vol. 37, No. 19, Applied Optics, (Jul. 1, 1998). [cited by applicant]
Tanriverdi, V. et al. (Apr. 2000). “Interacting With Eye Movements In Virtual Environments,” Department of Electrical Engineering and Computer Science, Tufts University, Medford, MA 02155, USA, Proceedings of the SIGCHI… [cited by applicant]
Yoshida, A. et al., “Design and Applications of a High Resolution Insert Head Mounted Display”, (Jun. 1994). [cited by applicant]
Japanese Notice of Allowance mailed Jan. 24, 2025, for JP Application No. 2024-071381, with English translation, 6 pages. [cited by applicant]