IP Library Granted Patent US 12,342,158
Granted Patent B2
US 12,342,158 · App. 18/761,089 · Granted Jun 24, 2025

Near-field audio rendering

Inventors: Remi Samuel Audfray (San Francisco, CA); Jean-Marc Jot (Aptos, CA); Samuel Charles Dicker (San Francisco, CA); Mark Brandon Hertensteiner (San Jose, CA); Justin Dan Mathew (Fort Lauderdale, FL); Anastasia Andreyevna Tajik (Fort Lauderdale, FL); Nicholas John LaMartina (Portland, ME)
Assignee: Magic Leap, Inc.
H04S7/304H04R5/033H04R5/04H04S3/008H04S2400/01H04S2400/11H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,342,158
App. No.
18/761,089
Granted
Jun 24, 2025
Kind
B2
Abstract

Examples of the disclosure describe systems and methods for presenting an audio signal to a user of a wearable head device. According to an example method, a source location corresponding to the audio signal is identified. For each of the respective left and right ear of the user, a virtual speaker position, of a virtual speaker array, is determined, the virtual speaker position collinear with the source location and with a position of the respective ear. For each of the respective left and right ear of the user, a head-related transfer function (HRTF) corresponding to the virtual speaker position and to the respective ear is determined; and the output audio signal is presented to the respective ear of the user via one or more speakers associated with the wearable head device. Processing the audio signal includes applying the HRTF to the audio signal.

Claims (44)

1. A method of presenting an audio signal to a user of a wearable head device, the method comprising:

determining a reference point based on a location of the wearable head device; and

for each of a respective left ear and right ear of the user:

determining, for a virtual speaker array associated with a sphere concentric with the reference point, the sphere having a first radius, a virtual speaker position substantially collinear with a source location and a position of the respective ear, wherein the determined virtual speaker position comprises a location substantially on a surface of the sphere and the source location corresponds to the audio signal;

determining a transfer function corresponding to the virtual speaker position;

generating, based on the transfer function and based further on the audio signal, an output audio signal for the respective ear;

attenuating the audio signal based on a distance between the source location and the respective ear, wherein the distance is clamped at a minimum value; and

presenting the output audio signal to the respective ear of the user via one or more speakers.

2. The method of claim 1 , wherein the source location is separated from the reference point by a distance less than the first radius.

3. The method of claim 1 , wherein the source location is separated from the reference point by a distance greater than the first radius.

4. The method of claim 1 , wherein the source location is separated from the reference point by a distance equal to the first radius.

5. The method of claim 1 , wherein said generating the output audio signal comprises applying an interaural time difference to the audio signal.

6. The method of claim 1 , wherein said determining the transfer function corresponding to the virtual speaker position comprises selecting the transfer function from a plurality of transfer functions, wherein each transfer function of the plurality of transfer functions describes a relationship between a listener and an audio source, the audio source separated from the listener by a distance substantially equal to the first radius.

7. The method of claim 1 , wherein the wearable head device comprises the one or more speakers.

8. The method of claim 1 , further comprising identifying the source location corresponding to the audio signal.

9. A system comprising:

a wearable head device;

one or more speakers; and

one or more processors configured to perform a method comprising:

determining a reference point based on a location of the wearable head device; and

for each of a respective left ear and right ear of the user:

determining, for a virtual speaker array associated with a sphere concentric with the reference point, the sphere having a first radius, a virtual speaker position substantially collinear with a source location and a position of the respective ear, wherein the determined virtual speaker position comprises a location substantially on a surface of the sphere and the source location corresponds to the audio signal;

determining a transfer function corresponding to the virtual speaker position;

generating, based on the transfer function and based further on the audio signal, an output audio signal for the respective ear;

attenuating the audio signal based on a distance between the source location and the respective ear, wherein the distance is clamped at a minimum value; and

presenting the output audio signal to the respective ear of the user via the one or more speakers.

10. The system of claim 9 , wherein the source location is separated from the reference point by a distance less than the first radius.

11. The system of claim 9 , wherein the source location is separated from the reference point by a distance greater than the first radius.

12. The system of claim 9 , wherein the source location is separated from the reference point by a distance equal to the first radius.

13. The system of claim 9 , wherein said generating the output audio signal comprises applying an interaural time difference to the audio signal.

14. The system of claim 9 , wherein said determining the transfer function corresponding to the virtual speaker position comprises selecting the transfer function from a plurality of transfer functions, wherein each transfer function of the plurality of transfer functions describes a relationship between a listener and an audio source, the audio source separated from the listener by a distance substantially equal to the first radius.

15. The system of claim 9 , wherein the wearable head device comprises the one or more speakers.

16. The system of claim 9 , the method further comprising identifying the source location corresponding to the audio signal.

17. A non-transitory computer-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform a method of presenting an audio signal to a user of a wearable head device, the method comprising:

determining a reference point based on a location of the wearable head device; and

for each of a respective left ear and right ear of the user:

determining, for a virtual speaker array associated with a sphere concentric with the reference point, the sphere having a first radius, a virtual speaker position substantially collinear with a source location and a position of the respective ear, wherein the determined virtual speaker position comprises a location substantially on a surface of the sphere and the source location corresponds to the audio signal;

determining a transfer function corresponding to the virtual speaker position;

generating, based on the transfer function and based further on the audio signal, an output audio signal for the respective ear;

attenuating the audio signal based on a distance between the source location and the respective ear, wherein the distance is clamped at a minimum value; and

presenting the output audio signal to the respective ear of the user via one or more speakers.

18. The non-transitory computer-readable medium of claim 17 , wherein the source location is separated from the reference point by a distance equal to the first radius.

19. The non-transitory computer-readable medium of claim 17 , wherein said generating the output audio signal comprises applying an interaural time difference to the audio signal.

20. The non-transitory computer-readable medium of claim 17 , wherein the wearable head device comprises the one or more speakers.

Assignments (3)
SECURITY INTEREST Recorded Nov 3, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073480/0540 →
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073387/0487 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 13, 2024
From: AUDFRAY, REMI SAMUEL; JOT, JEAN-MARC; DICKER, SAMUEL CHARLES; HERTENSTEINER, MARK BRANDON; MATHEW, JUSTIN DAN; TAJIK, ANASTASIA ANDREYEVNA; LAMARTINA, NICHOLAS JOHN
To: MAGIC LEAP, INC.
Reel/Frame 069253/0786 →
Continuity (7)
Continuation 18451794 · Aug 17, 2023
Continuation 18061367 · Dec 2, 2022
Continuation 17401090 · Aug 12, 2021
Continuation 16593943 · Oct 4, 2019
Provisional Application 62812734 · Mar 1, 2019
Provisional Application 62741677 · Oct 5, 2018
Related Publication 20240357311A1 · Oct 24, 2024
References Cited (126)
US 4852988A · Velez · 1989 [cited by applicant]
US 5596644A · Abel et al. · 1997 [cited by applicant]
US 6433760B1 · Vaissie · 2002 [cited by applicant]
US 6491391B1 · Blum et al. · 2002 [cited by applicant]
US 6546105B1 · Katayama et al. · 2003 [cited by applicant]
US 6819762B2 · Jones et al. · 2004 [cited by applicant]
US 6847336B1 · Lemelson · 2005 [cited by applicant]
US 6943754B2 · Aughey · 2005 [cited by applicant]
US 6977776B2 · Volkenandt et al. · 2005 [cited by applicant]
US 7347551B2 · Fergason et al. · 2008 [cited by applicant]
US 7488294B2 · Torch · 2009 [cited by applicant]
US 8235529B1 · Raffle · 2012 [cited by applicant]
US 8611015B2 · Wheeler · 2013 [cited by applicant]
US 8638498B2 · Bohn et al. · 2014 [cited by applicant]
US 8696113B2 · Lewis · 2014 [cited by applicant]
US 8767968B2 · Flaks et al. · 2014 [cited by applicant]
US 8929589B2 · Publicover et al. · 2015 [cited by applicant]
US 9010929B2 · Lewis · 2015 [cited by applicant]
US 9274338B2 · Robbins et al. · 2016 [cited by applicant]
US 9292973B2 · Bar-zeev et al. · 2016 [cited by applicant]
US 9323325B2 · Perez et al. · 2016 [cited by applicant]
US 9720505B2 · Gribetz et al. · 2017 [cited by applicant]
US 9955281B1 · Lyren et al. · 2018 [cited by applicant]
US 9992602B1 · Allen · 2018 [cited by applicant]
US 10013053B2 · Cederlund et al. · 2018 [cited by applicant]
US 10025379B2 · Drake et al. · 2018 [cited by applicant]
US 11122383B2 · Audfray et al. · 2021 [cited by applicant]
US 11212636B2 · Tajik · 2021 [cited by applicant]
US 11546716B2 · Audfray et al. · 2023 [cited by applicant]
US 11589182B2 · Tajik · 2023 [cited by applicant]
US 11736888B2 · Tajik · 2023 [cited by applicant]
US 11778411B2 · Audfray et al. · 2023 [cited by applicant]
US 11956620B2 · Tajik · 2024 [cited by applicant]
US 12063497B2 · Audfray · 2024 [cited by applicant]
US 20030030597A1 · Geist · 2003 [cited by applicant]
US 20060023158A1 · Howell et al. · 2006 [cited by applicant]
US 20090003614A1 · Neunaber · 2009 [cited by applicant]
US 20090180624A1 · Nakayama · 2009 [cited by applicant]
US 20100080396A1 · Aoyagi · 2010 [cited by applicant]
US 20110211056A1 · Publicover et al. · 2011 [cited by applicant]
US 20110213664A1 · Osterhout · 2011 [cited by applicant]
US 20120021806A1 · Maltz · 2012 [cited by applicant]
US 20120093320A1 · Flaks et al. · 2012 [cited by applicant]
US 20120206452A1 · Geisner et al. · 2012 [cited by applicant]
US 20130077147A1 · Efimov · 2013 [cited by applicant]
US 20130120224A1 · Cajigas · 2013 [cited by applicant]
US 20130236040A1 · Crawford · 2013 [cited by applicant]
US 20140195918A1 · Friedlander · 2014 [cited by applicant]
US 20140320389A1 · Scavezze · 2014 [cited by applicant]
US 20150168731A1 · Robbins · 2015 [cited by applicant]
US 20150373474A1 · Kraft et al. · 2015 [cited by applicant]
US 20160025982A1 · Sutherland · 2016 [cited by applicant]
US 20160134987A1 · Gorzel et al. · 2016 [cited by applicant]
US 20160163110A1 · Chang et al. · 2016 [cited by applicant]
US 20170153866A1 · Grinberg · 2017 [cited by applicant]
US 20170223478A1 · Jot · 2017 [cited by applicant]
US 20180101990A1 · Yang et al. · 2018 [cited by applicant]
US 20180109901A1 · Laaksonen · 2018 [cited by applicant]
US 20180321894A1 · Paulovich et al. · 2018 [cited by applicant]
US 20180341455A1 · Ivanov · 2018 [cited by applicant]
US 20190102956A1 · Ishihara · 2019 [cited by applicant]
US 20190293609A1 · Oh · 2019 [cited by applicant]
US 20190313201A1 · Torres et al. · 2019 [cited by applicant]
US 20240205630A1 · Tajik · 2024 [cited by applicant]
CA 2316473A1 · 2001 [cited by applicant]
CA 2362895A1 · 2002 [cited by applicant]
CA 2388766A1 · 2003 [cited by applicant]
GB 2536020A · 2016 [cited by applicant]
WO 2015072091A1 · 2015 [cited by applicant]
WO 2015192117A1 · 2015 [cited by applicant]
WO 2017183346A1 · 2017 [cited by applicant]
WO 2018132235A1 · 2018 [cited by applicant]
Azuma, Ronald T. (Aug. 1997). “A Survey of Augmented Reality,” In Presence: Teleoperators and Virtual Environments 6, 4, Hughes Research Laboratories, Malibu, CA, located at: https://web.archive.org/web/20010604100006/h… [cited by applicant]
Carmigniani, J., et al., “Augmented Reality Technologies, Systems and Applications,” Multimedia Tools and Applications, 51(1):341-77 (Dec. 2010) (“Augmented Reality Technologies”). [cited by applicant]
Declaration of Dr. Stephen C. Thompson Under 37 C.F.R. § 1.132 including Appendix A (List of Materials Considered) and Appendix B (Curriculum Vitae) and Incorporated Claim Charts CC-1-CC-9. [cited by applicant]
Feiner, S., “The Importance of Being Mobile: Some Social Consequences of Wearable Augmented Reality Systems,” Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99) (Oct. 1999). [cited by applicant]
Milgram, P. and Kishino, F., “A Taxonomy of Mixed Reality Visual Displays,” IEICE Transactions on Information & Systems, E77-D(12):1321-29 (Dec. 1994) (“Taxonomy of Mixed Reality”). [cited by applicant]
Non-Final Office Action mailed Sep. 18, 2024, for U.S. Appl. No. 18/587,728, filed Feb. 26, 2024, thirty-three pages. [cited by applicant]
Prosecution History of U.S. Pat. No. 11,212,636. [cited by applicant]
Chinese Notice of Allowance dated Jan. 16, 2023, for CN Application No. 201980080065.2, with English translation, 6 pages. [cited by applicant]
Chinese Notice of Allowance dated Jan. 26, 2022, for CN Application No. 201980012942.2, with English translation, six pages. [cited by applicant]
Chinese Notice of Allowance dated Jan. 4, 2024, for CN Application No. 202210301214.7, with English translation, six pages. [cited by applicant]
Chinese Office Action dated Jul. 16, 2021, for CN Application No. 201980012942.2, with English translation, 16 pages. [cited by applicant]
Chinese Office Action dated Jul. 20, 2022, for CN Application No. 201980080065.2, with English translation 15 pages. [cited by applicant]
Chinese Office Action dated Sep. 1, 2023, for CN Application No. 202210301214.7, with English translation, 7 pages. [cited by applicant]
European Office Action dated Oct. 10, 2023, for EP Application No. 19869249.3, six pages. [cited by applicant]
European Search Report mailed Nov. 30, 2021, for EP Application No. 19869249.3, six pages. [cited by applicant]
Final Office Action mailed Nov. 12, 2020, for U.S. Appl. No. 16/593,943, filed Oct. 4, 2019, 17 pages. [cited by applicant]
International Preliminary Report on Patentability and Written Opinion dated Apr. 15, 2021, for PCT Application No. PCT/US2019/054893, filed Oct. 4, 2019, eleven pages. [cited by applicant]
International Preliminary Report on Patentability dated Aug. 18, 2020, for PCT Application No. PCT/US2019/018369, filed Feb. 15, 2019, five pages. [cited by applicant]
International Search Report and Written Opinion, mailed May 7, 2019, for PCT Application No. PCT/US2019/18369, filed Feb. 15, 2019, eleven pages. [cited by applicant]
International Search Report dated Jan. 10, 2020, for PCT Application No. PCT/US2019/054893, filed Oct. 4, 2019, one page. [cited by applicant]
Israeli Notice of Allowance dated Dec. 22, 2022, for IL Application No. 276496, three pages. [cited by applicant]
Israeli Notice of Allowance dated Jul. 12, 2023, for IL Application No. 301445, four pages. [cited by applicant]
Israeli Notice of Allowance dated Jul. 9, 2024, for IL Application No. 307545, four pages. [cited by applicant]
Israeli Notice of Allowance dated Jun. 16, 2024, for IL Application No. 307545, three pages. [cited by applicant]
Jacob, R. “Eye Tracking in Advanced Interface Design”, Virtual Environments and Advanced Interface Design, Oxford University Press, Inc. (Jun. 1995). [cited by applicant]
Japanese Notice of Allowance mailed Dec. 28, 2023, for JP Application No. 2022-196982, with English translation, six pages. [cited by applicant]
Yoshida, A. et al., “Design and Applications of a High Resolution Insert Head Mounted Display”, (Jun. 1994). [cited by applicant]
Japanese Notice of Allowance mailed Nov. 22, 2024, for JP Application No. 2023-047790, with English translation, 6 pages. [cited by applicant]
Ex Parte Reexamination Communication mailed Mar. 20, 2025, for U.S. Appl. No. 90/019,640, filed Aug. 29, 2024, for U.S. Pat. No. 11,212,636, forty-one pages. [cited by applicant]
Japanese Notice of Allowance mailed Mar. 24, 2025, for JP Application No. 2024037946, with English translation, 6 pages. [cited by applicant]
Notice of Allowance mailed Mar. 27, 2025, for U.S. Appl. No. 18/587,728, filed Feb. 26, 2024, nineteen pages. [cited by applicant]
Japanese Notice of Allowance mailed Apr. 9, 2025, for JP Application No. 2024-135322, with English translation, 6 pages. [cited by applicant]
Japanese Notice of Allowance mailed Feb. 16, 2024, for JP Application No. 2022-160027, with English translation, six pages. [cited by applicant]
Japanese Notice of Allowance mailed Feb. 28, 2023, for JP Application No. 2020-543356, with English translation, 6 pages. [cited by applicant]
Japanese Notice of Allowance mailed Nov. 10, 2022, for JP Application No. 2021-518639, with English translation, six pages. [cited by applicant]
Japanese Office Action mailed May 29, 2024, for JP Application No. 2023-047790, with English translation, 8 pages. [cited by applicant]
Japanese Office Action mailed Oct. 20, 2023, for JP Application No. 2022-160027, with English translation, 4 pages. [cited by applicant]
Non-Final Office Action mailed Apr. 13, 2023, for U.S. Appl. No. 18/061,367, filed Dec. 2, 2022, nine pages. [cited by applicant]
Non-Final Office Action mailed Jul. 7, 2020, for U.S. Appl. No. 16/593,943, filed Oct. 4, 2019, 13 pages. [cited by applicant]
Non-Final Office Action mailed Mar. 28, 2024, for U.S. Appl. No. 18/451,794, filed Aug. 17, 2023, five pages. [cited by applicant]
Notice of Allowability (corrected) mailed Jun. 18, 2021, for U.S. Appl. No. 16/593,943, filed Oct. 4, 2019, 6 pages. [cited by applicant]
Notice of Allowance (corrected) mailed Feb. 14, 2024, for U.S. Appl. No. 18/340,744, filed Jun. 23, 2023, twelve pages. [cited by applicant]
Notice of Allowance mailed Aug. 18, 2021, for U.S. Appl. No. 16/970,324, filed Aug. 14, 2020, 11 pages. [cited by applicant]
Notice of Allowance mailed Jan. 26, 2024, for U.S. Appl. No. 18/340,744, filed Jun. 23, 2023, fifteen pages. [cited by applicant]
Notice of Allowance mailed Jul. 20, 2023, for U.S. Appl. No. 18/061,367, filed Dec. 2, 2022, five pages. [cited by applicant]
Notice of Allowance mailed Mar. 29, 2021, for U.S. Appl. No. 16/593,943, filed Oct. 4, 2019, eight pages. [cited by applicant]
Notice of Allowance mailed May 24, 2023, for U.S. Appl. No. 18/047,168, filed Oct. 17, 2022, eighteen pages. [cited by applicant]
Notice of Allowance mailed May 30, 2024, for U.S. Appl. No. 18/451,794, filed Aug. 17, 2023, five pages. [cited by applicant]
Notice of Allowance mailed Sep. 16, 2022, for U.S. Appl. No. 17/528,473, filed Nov. 17, 2021, 13 pages. [cited by applicant]
Notice of Allowance mailed Sep. 7, 2022, for U.S. Appl. No. 17/401,090, filed Aug. 12, 2021, five pages. [cited by applicant]
Olson, H. F. (Mar. 1973). “Gradient loudspeakers,” RCA Laboratories, Princeton, N. J., Journal of the Audio Engineering Society, vol. 21, 86-93. [cited by applicant]
Rolland, J. et al., “High-resolution inset head-mounted display”, Optical Society of America, vol. 37, No. 19, Applied Optics, (Jul. 1, 1998). [cited by applicant]
Tanriverdi, V. et al. (Apr. 2000). “Interacting With Eye Movements In Virtual Environments,” Department of Electrical Engineering and Computer Science, Tufts University, Medford, MA 02155, USA, Proceedings of the SIGCHI… [cited by applicant]
Wenzel, E. M., et al.: “Sound Lab: A real-time, software-based system for the study of spatial hearing”, Internet Citation, Feb. 19, 2000 (Feb. 19, 2000), Retrieved from the Internet: URL:http://pddocserv/specdocs/data/… [cited by applicant]