IP Library Granted Patent US 12,308,011
Granted Patent B2
US 12,308,011 · App. 18/653,795 · Granted May 20, 2025

Reverberation gain normalization

Inventors: Remi Samuel Audfray (San Francisco, CA); Jean-Marc Jot (Aptos, CA); Samuel Charles Dicker (San Francisco, CA)
Assignee: Magic Leap, Inc.
G10K15/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,308,011
App. No.
18/653,795
Granted
May 20, 2025
Kind
B2
Abstract

Systems and methods for providing accurate and independent control of reverberation properties are disclosed. In some embodiments, a system may include a reverberation processing system, a direct processing system, and a combiner. The reverberation processing system can include a reverb initial power (RIP) control system and a reverberator. The RIP control system can include a reverb initial gain (RIG) and a RIP corrector. The RIG can be configured to apply a RIG value to the input signal, and the RIP corrector can be configured to apply a RIP correction factor to the signal from the RIG. The reverberator can be configured to apply reverberation effects to the signal from the RIP control system. In some embodiments, one or more values and/or correction factors can be calculated and applied such that the signal output from a component in the reverberation processing system is normalized to a predetermined value (e.g., unity (1.0)).

Claims (52)

1. A method comprising:

receiving an input signal;

applying a reverb gain (RG) value to the input signal;

applying a reverb energy (RE) correction factor to the input signal;

applying a reverb initial power (RIP) correction factor to the input signal;

determining an audio signal based on the application of the RG value, the RE correction factor, and the RIP correction factor to the input signal; and

presenting the audio signal to via one or more speakers of a wearable head device.

2. The method of claim 1 , further comprising:

determining the RIP correction factor, wherein the RIP correction factor is determined and applied to the input signal by a RIP corrector,

wherein the RIP correction factor is determined such that a signal output from the RIP corrector is normalized.

3. The method of claim 1 , wherein the RIP correction factor is based on one or more of: a reverberator topology, a number of delay units, a delay duration, a connection gain, and a filter parameter.

4. The method of claim 1 , wherein the RIP correction factor is based on a power of a reverberation impulse response.

5. The method of claim 1 , wherein the applying the RIP correction factor comprises selecting a reverberator topology and setting internal reverberator parameters.

6. The method of claim 1 , wherein the RE correction factor is based on a decay time, a reverb energy, and further based on the RIP correction factor such that a RIP corrected signal is normalized.

7. The method of claim 1 , further comprising determining the RIP correction factor, wherein:

the determining the RIP correction factor comprises:

determining a reverberator impulse response, and

determining a reverberation RMS amplitude, and

the RIP correction factor is inversely related to the reverberation RMS amplitude.

8. The method of claim 7 , wherein the determining the RIP correction factor further comprises determining a reverberation RMS amplitude decay.

9. The method of claim 1 , further comprising:

determining the RE correction factor, wherein the RE correction factor is applied to the input signal by a RE corrector,

wherein the RE correction factor is determined such that a signal output from the RE corrector is normalized.

10. The method of claim 1 , further comprising applying a reverberation effect to the input signal, wherein the audio signal is further based on the application of the reverberation effect.

11. The method of claim 10 , wherein the applying the reverberation effect comprises filtering one or more frequencies of the input signal.

12. The method of claim 10 , wherein the applying the reverberation effect comprises changing a phase of the input signal.

13. The method of claim 10 , wherein the reverberation effect is applied after the RIP correction factor is applied.

14. The method of claim 1 , wherein one or more of the RG value, RE correction factor, and RIP correction factor are determined via one or more sensors of the wearable head device.

15. A system comprising:

a wearable head device comprising one or more speakers, the wearable head device configured to present an audio signal to a user; and

one or more processors configured to perform a method comprising:

receiving an input signal;

applying a reverb gain (RG) value to the input signal;

applying a reverb energy (RE) correction factor to the input signal;

applying a reverb initial power (RIP) correction factor to the input signal;

determining the audio signal based on the application of the RG value, the RE correction factor, and the RIP correction factor to the input signal; and

presenting the audio signal to via the one or more speakers.

16. The system of claim 15 , wherein the method further comprises:

determining the RIP correction factor, wherein the RIP correction factor is determined and applied to the input signal by a RIP corrector,

wherein the RIP correction factor is determined such that a signal output from the RIP corrector is normalized.

17. The system of claim 15 , wherein the method further comprises applying a reverberation effect to the input signal, wherein the audio signal is further based on the application of the reverberation effect.

18. The system of claim 15 , wherein:

the wearable head device comprises one or more sensors, and

one or more of the RG value, RE correction factor, and RIP correction factor are determined via the one or more sensors.

19. A non-transitory computer-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform a method comprising:

receiving an input signal;

applying a reverb gain (RG) value to the input signal;

applying a reverb energy (RE) correction factor to the input signal;

applying a reverb initial power (RIP) correction factor to the input signal;

determining an audio signal based on the application of the RG value, the RE correction factor, and the RIP correction factor to the input signal; and

presenting the audio signal to via one or more speakers of a wearable head device.

20. The non-transitory computer-readable medium of claim 19 , wherein one or more of the RG value, RE correction factor, and RIP correction factor are determined via one or more sensors of the wearable head device.

Assignments (3)
SECURITY INTEREST Recorded Oct 31, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073439/0168 →
SECURITY INTEREST Recorded Oct 24, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073255/0581 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 6, 2024
From: AUDFRAY, REMI SAMUEL; JOT, JEAN-MARC; DICKER, SAMUEL CHARLES
To: MAGIC LEAP, INC.
Reel/Frame 067324/0419 →
Continuity (6)
Continuation 18296901 · Apr 6, 2023
Continuation 17568588 · Jan 4, 2022
Continuation 17020584 · Sep 14, 2020
Continuation 16442359 · Jun 14, 2019
Provisional Application 62685235 · Jun 14, 2018
Related Publication 20240282289A1 · Aug 22, 2024
References Cited (150)
US 4852988A · Velez · 1989 [cited by applicant]
US 4953219A · Kasai et al. · 1990 [cited by applicant]
US 5555306A · Gerzon · 1996 [cited by applicant]
US 6433760B1 · Vaissie · 2002 [cited by applicant]
US 6491391B1 · Blum et al. · 2002 [cited by applicant]
US 6847336B1 · Lemelson · 2005 [cited by applicant]
US 6943754B2 · Aughey · 2005 [cited by applicant]
US 6977776B2 · Volkenandt et al. · 2005 [cited by applicant]
US 7347551B2 · Fergason et al. · 2008 [cited by applicant]
US 7488294B2 · Torch · 2009 [cited by applicant]
US 8235529B1 · Raffle · 2012 [cited by applicant]
US 8611015B2 · Wheeler · 2013 [cited by applicant]
US 8638498B2 · Bohn et al. · 2014 [cited by applicant]
US 8696113B2 · Lewis · 2014 [cited by applicant]
US 8929589B2 · Publicover et al. · 2015 [cited by applicant]
US 9010929B2 · Lewis · 2015 [cited by applicant]
US 9274338B2 · Robbins et al. · 2016 [cited by applicant]
US 9292973B2 · Bar-zeev et al. · 2016 [cited by applicant]
US 9323325B2 · Perez et al. · 2016 [cited by applicant]
US 9432790B2 · Raghuvanshi et al. · 2016 [cited by applicant]
US 9558757B1 · Li · 2017 [cited by applicant]
US 9720505B2 · Gribetz et al. · 2017 [cited by applicant]
US 9906885B2 · Visser et al. · 2018 [cited by applicant]
US 9940922B1 · Schissler et al. · 2018 [cited by applicant]
US 10013053B2 · Cederlund et al. · 2018 [cited by applicant]
US 10025379B2 · Drake et al. · 2018 [cited by applicant]
US 10038967B2 · Jot · 2018 [cited by applicant]
US 10440498B1 · Amengual Garí et al. · 2019 [cited by applicant]
US 10531220B2 · Sanger et al. · 2020 [cited by applicant]
US 10609502B2 · Zhang et al. · 2020 [cited by applicant]
US 10645520B1 · Amengual Gari et al. · 2020 [cited by applicant]
US 10674307B1 · Robinson et al. · 2020 [cited by applicant]
US 10685641B2 · Asada et al. · 2020 [cited by applicant]
US 10735884B2 · Audfray et al. · 2020 [cited by applicant]
US 10810992B2 · Audfray · 2020 [cited by applicant]
US 10880668B1 · Robinson et al. · 2020 [cited by applicant]
US 10952010B2 · Audfray et al. · 2021 [cited by applicant]
US 11184727B2 · Jeon et al. · 2021 [cited by applicant]
US 11250834B2 · Audfray et al. · 2022 [cited by applicant]
US 11304017B2 · Parvaix et al. · 2022 [cited by applicant]
US 11540072B2 · Parvaix et al. · 2022 [cited by applicant]
US 11651762B2 · Audfray et al. · 2023 [cited by applicant]
US 11778398B2 · Parvaix et al. · 2023 [cited by applicant]
US 12008982B2 · Audfray · 2024 [cited by applicant]
US 12149896B2 · Parvaix · 2024 [cited by applicant]
US 20030030597A1 · Geist · 2003 [cited by applicant]
US 20030159569A1 · Ohta · 2003 [cited by applicant]
US 20060023158A1 · Howell et al. · 2006 [cited by applicant]
US 20070160241A1 · Joublin · 2007 [cited by applicant]
US 20070173730A1 · Bikko · 2007 [cited by applicant]
US 20070195967A1 · Wu · 2007 [cited by applicant]
US 20080275697A1 · Kentish · 2008 [cited by applicant]
US 20090220100A1 · Ohta et al. · 2009 [cited by applicant]
US 20110211056A1 · Publicover et al. · 2011 [cited by applicant]
US 20110213664A1 · Osterhout · 2011 [cited by applicant]
US 20110235813A1 · Gauger, Jr. · 2011 [cited by applicant]
US 20110268283A1 · Nakadai · 2011 [cited by applicant]
US 20120021806A1 · Maltz · 2012 [cited by applicant]
US 20130077147A1 · Efimov · 2013 [cited by applicant]
US 20140003630A1 · Demiya · 2014 [cited by applicant]
US 20140153727A1 · Walsh et al. · 2014 [cited by applicant]
US 20140195918A1 · Friedlander · 2014 [cited by applicant]
US 20140270184A1 · Beaton · 2014 [cited by applicant]
US 20150168731A1 · Robbins · 2015 [cited by applicant]
US 20150379980A1 · Neugebauer · 2015 [cited by applicant]
US 20150380010A1 · Srinivasan · 2015 [cited by applicant]
US 20160057522A1 · Choisel et al. · 2016 [cited by applicant]
US 20170041445A1 · Ukai et al. · 2017 [cited by applicant]
US 20170080229A1 · Meister · 2017 [cited by applicant]
US 20170126194A1 · Jot · 2017 [cited by applicant]
US 20170134717A1 · Trail · 2017 [cited by applicant]
US 20170223478A1 · Jot · 2017 [cited by applicant]
US 20170238119A1 · Schlecht et al. · 2017 [cited by applicant]
US 20180020312A1 · Visser et al. · 2018 [cited by applicant]
US 20180091486A1 · Liu · 2018 [cited by applicant]
US 20180206046A1 · Rosenkranz et al. · 2018 [cited by applicant]
US 20180359582A1 · Belhomme et al. · 2018 [cited by applicant]
US 20190116448A1 · Schmidt · 2019 [cited by applicant]
US 20190272842A1 · Bryan · 2019 [cited by applicant]
US 20190387352A1 · Jot et al. · 2019 [cited by applicant]
US 20200046265A1 · Kaifosh · 2020 [cited by applicant]
US 20200089466A1 · Eronen · 2020 [cited by applicant]
US 20200260209A1 · Yadegari · 2020 [cited by applicant]
US 20200314583A1 · Robinson · 2020 [cited by applicant]
US 20200327879A1 · Tsuji et al. · 2020 [cited by applicant]
US 20200381007A1 · Hur et al. · 2020 [cited by applicant]
US 20200382895A1 · Schissler · 2020 [cited by applicant]
US 20200396560A1 · Family · 2020 [cited by applicant]
US 20210084357A1 · Tajik et al. · 2021 [cited by applicant]
US 20210110809A1 · Lastrucci · 2021 [cited by applicant]
US 20230403524A1 · Parvaix et al. · 2023 [cited by applicant]
US 20240414492A1 · Hertensteiner et al. · 2024 [cited by applicant]
US 20250039622A1 · Parvaix et al. · 2025 [cited by applicant]
CA 2316473A1 · 2001 [cited by applicant]
CA 2362895A1 · 2002 [cited by applicant]
CA 2388766A1 · 2003 [cited by applicant]
CN 103675104A · 2014 [cited by applicant]
JP S5264974A · 1977 [cited by applicant]
JP 2003330477A · 2003 [cited by applicant]
JP 2005012784A · 2005 [cited by applicant]
JP 2009139615A · 2009 [cited by applicant]
JP 2011232691A · 2011 [cited by applicant]
JP 2014026241A · 2014 [cited by applicant]
JP 2019527956A · 2019 [cited by applicant]
JP 2019549205 · 2020 [cited by applicant]
WO 2019078034A1 · 2019 [cited by applicant]
Chinese Office Action dated Apr. 29, 2024, for CN Patent Application No. 201980052745.3, with English translation, 8 pages. [cited by applicant]
Chinese Office Action dated Dec. 11, 2023, for CN Patent Application No. 201980052745.3, with English translation, fifteen pages. [cited by applicant]
European Office Action mailed Dec. 15, 2022, for EP Application No. 19820590.8, four pages. [cited by applicant]
European Search Report dated Jul. 15, 2024, for EP Application No. 24167861.4, seven pages. [cited by applicant]
European Search Report dated Nov. 24, 2022, for EP Application No. 20880077.1, nine pages. [cited by applicant]
European Search Report mailed Jul. 9, 2021, for EP Application No. 19820590.8, ten pages. [cited by applicant]
Ex Parte Quayle Office Action mailed Nov. 8, 2021, for U.S. Appl. No. 17/079,212, filed Oct. 23, 2020, 12 pages. [cited by applicant]
Final Office Action mailed Apr. 8, 2020, for U.S. Appl. No. 16/442,359, filed Jun. 14, 2019, 15 pages. [cited by applicant]
International Preliminary Report on Patentability and Written Opinion dated Dec. 15, 2020, for PCT Application No. PCT/US19/37384, filed Jun. 14, 2019, seven pages. [cited by applicant]
International Preliminary Report on Patentability and Written Opinion, mailed May 5, 2022, for PCT Application No. PCT/US2020/057203, filed Oct. 23, 2020, six pages. [cited by applicant]
International Preliminary Report on Patentability mailed May 10, 2024, for PCT Application No. PCT/US2022/078316, filed Oct. 18, 2022, twelve pages. [cited by applicant]
International Search Report and Written Opinion mailed Feb. 28, 2023, for PCT Application No. PCT/US2022/078316, filed Oct. 18, 2022, 22 pages. [cited by applicant]
International Search Report and Written Opinion, mailed Jan. 26, 2021, for PCT Application No. PCT/US2020/057203, filed Oct. 23, 2020, eleven pages. [cited by applicant]
International Search Report mailed Sep. 13, 2019, for PCT Application No. PCT/US19/37384, filed Jun. 14, 2019, three pages. [cited by applicant]
Jacob, R. “Eye Tracking in Advanced Interface Design”, Virtual Environments and Advanced Interface Design, Oxford University Press, Inc. (Jun. 1995). [cited by applicant]
Japanese Notice of Allowance mailed Apr. 9, 2024, for JP Application No. 2020-569075, with English translation, 6 pages. [cited by applicant]
Japanese Notice of Allowance mailed Feb. 19, 2024, for JP Application No. 2022-523600, with English translation, six pages. [cited by applicant]
Japanese Office Action mailed Jul. 7, 2023, for JP Application No. 2020-569075, with English translation, 12 pages. [cited by applicant]
Japanese Office Action mailed Jun. 12, 2024, for JP Application No. 2023-214712, with English translation, 11 pages. [cited by applicant]
Japanese Office Action mailed Nov. 6, 2023, for JP Application No. 2022-523600, with English translation, six pages. [cited by applicant]
Jot, J-M. et al. (Oct. 2016). “Augmented Reality Headphone Environment Rendering,” Audio Eng. Soc. Conf. on Audio for Virtual and Augmented Reality, Los Angeles, California, Sep. 30-Oct. 1, 2016, six pages. [cited by applicant]
Murgai, P. et al. (Oct. 2017). “Blind Estimation of the Reverberation Fingerprint of Unknown Acoustic Environments,” Convention Paper 9905, Audio Engineering Society, 143rd Conv. New York, New York, Oct. 18-21, 2017, si… [cited by applicant]
Non-Final Office Action mailed Jun. 10, 2021, for U.S. Appl. No. 17/020,584, filed Sep. 14, 2020, ten pages. [cited by applicant]
Non-Final Office Action mailed Mar. 8, 2023, for U.S. Appl. No. 18/058,232, filed Nov. 22, 2022, six pages. [cited by applicant]
Non-Final Office Action mailed May 17, 2024, for U.S. Appl. No. 18/237,803, filed Aug. 24, 2023, six pages. [cited by applicant]
Non-Final Office Action mailed Oct. 23, 2023, for U.S. Appl. No. 18/296,901, filed Apr. 6, 2023, 16 pages. [cited by applicant]
Non-Final Office Action mailed Oct. 30, 2019, for U.S. Appl. No. 16/442,359, filed Jun. 14, 2019, eleven pages. [cited by applicant]
Non-Final Office Action mailed Sep. 26, 2022, for U.S. Appl. No. 17/568,588, filed Jan. 4, 2022, nine pages. [cited by applicant]
Notice of Allowance mailed Apr. 10, 2024, for U.S. Appl. No. 18/296,901, filed Apr. 6, 2023, seven pages. [cited by applicant]
Notice of Allowance mailed Feb. 7, 2022, for U.S. Appl. No. 17/079,212, filed Oct. 23, 2020, 5 pages. [cited by applicant]
Notice of Allowance mailed Jul. 27, 2023, for U.S. Appl. No. 18/058,232, filed Nov. 22, 2022, eight pages. [cited by applicant]
Notice of Allowance mailed Jun. 12, 2020, for U.S. Appl. No. 16/442,359, filed Jun. 14, 2019, ten pages. [cited by applicant]
Notice of Allowance mailed Mar. 9, 2023, for U.S. Appl. No. 17/568,588, filed Jan. 4, 2022, seven pages. [cited by applicant]
Notice of Allowance mailed Oct. 19, 2022, for U.S. Appl. No. 17/686,025, filed Mar. 3, 2022, eight pages. [cited by applicant]
Notice of Allowance mailed Oct. 5, 2021, for U.S. Appl. No. 17/020,584, filed Sep. 14, 2020, seven pages. [cited by applicant]
Rolland, J. et al., “High-resolution inset head-mounted display”, Optical Society of America, vol. 37, No. 19, Applied Optics, (Jul. 1, 1998). [cited by applicant]
Tanriverdi, V. et al. (Apr. 2000). “Interacting With Eye Movements In Virtual Environments,” Department of Electrical Engineering and Computer Science, Tufts University, Medford, MA 02155, USA, Proceedings of the SIGCHI… [cited by applicant]
Yoshida, A. et al., “Design and Applications of a High Resolution Insert Head Mounted Display”, (Jun. 1994). [cited by applicant]
Chinese Notice of Allowance dated Oct. 25, 2024, for CN Patent Application No. 201980052745.3, with English translation, six pages. [cited by applicant]
Chinese Office Action dated Nov. 11, 2024, for CN Application No. 202080074331.3, with English translation, 25 pages. [cited by applicant]
European Notice of Allowance dated Nov. 21, 2024, for EP Application No. 20880077.1, nine pages. [cited by applicant]
Japanese Final Office Action mailed Oct. 10, 2024, for JP Application No. 2023-214712, with English translation, 13 pages. [cited by applicant]
Japanese Notice of Allowance mailed Jan. 31, 2025, for JP Application No. 2023-214712, with English translation, 7 pages. [cited by applicant]
Notice of Allowance mailed Sep. 10, 2024, for U.S. Appl. No. 18/237,803, filed Aug. 24, 2023, eight pages. [cited by applicant]