IP Library Granted Patent US 11,122,383
Granted Patent B2
US 11,122,383 · App. 16/593,943 · Granted Sep 14, 2021

Near-field audio rendering

Inventors: Remi Samuel Audfray (San Francisco, CA); Jean-Marc Jot (Aptos, CA); Samuel Charles Dicker (San Francisco, CA); Mark Brandon Hertensteiner (San Jose, CA); Justin Dan Mathew (Fort Lauderdale, FL); Anastasia Andreyevna Tajik (Fort Lauderdale, FL); Nicholas John LaMartina (Portland, ME)
Assignee: Magic Leap, Inc.
H04S7/304H04R5/033H04R5/04H04S3/008H04S2400/01H04S2400/11H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,122,383
App. No.
16/593,943
Granted
Sep 14, 2021
Kind
B2
Abstract

Examples of the disclosure describe systems and methods for presenting an audio signal to a user of a wearable head device. According to an example method, a source location corresponding to the audio signal is identified. An acoustic axis corresponding to the audio signal is determined. For each of a respective left and right ear of the user, an angle between the acoustic axis and the respective ear is determined. For each of the respective left and right ear of the user, a virtual speaker position, of a virtual speaker array, is determined, the virtual speaker position collinear with the source location and with a position of the respective ear. The virtual speaker array includes a plurality of virtual speaker positions, each virtual speaker position of the plurality located on the surface of a sphere concentric with the user's head, the sphere having a first radius. For each of the respective left and right ear of the user, a head-related transfer function (HRTF) corresponding to the virtual speaker position and to the respective ear is determined; a source radiation filter is determined based on the determined angle; the audio signal is processed to generate an output audio signal for the respective ear; and the output audio signal is presented to the respective ear of the user via one or more speakers associated with the wearable head device. Processing the audio signal includes applying the HRTF and the source radiation filter to the audio signal.

Claims (72)

1. A method of presenting an audio signal to a user of a wearable head device, the method comprising:

identifying a source location corresponding to the audio signal;

determining an acoustic axis corresponding to the audio signal;

for each of a respective left and right ear of the user:

determining an angle between the acoustic axis and the respective ear;

determining, of a virtual speaker array, a virtual speaker position collinear with the source location and a position of the respective ear, wherein the virtual speaker array comprises a plurality of virtual speaker positions, each virtual speaker position of the plurality of virtual speaker positions located on the surface of a sphere concentric with the user's head, the sphere having a first radius;

determining a head-related transfer function (HRTF) corresponding to the virtual speaker position and to the respective ear;

determining a source radiation filter based on the determined angle;

processing the audio signal to generate an output audio signal for the respective ear, wherein processing the audio signal comprises applying the HRTF and the source radiation filter to the audio signal;

attenuating the audio signal based on a distance between the source location and the respective ear wherein the distance is clamped at a minimum value; and

presenting the output audio signal to the respective ear of the user via one or more speakers associated with the wearable head device.

2. The method of claim 1 , wherein the source location is separated from a center of the user's head by a distance less than the first radius.

3. The method of claim 1 , wherein the source location is separated from a center of the user's head by a distance greater than the first radius.

4. The method of claim 1 , wherein the source location is separated from a center of the user's head by a distance equal to the first radius.

5. The method of claim 1 , wherein processing the audio signal further comprises applying an interaural time difference to the audio signal.

6. The method of claim 1 , wherein determining the HRTF corresponding to the virtual speaker position comprises selecting the HRTF from a plurality of HRTFs, wherein each HRTF of the plurality of HRTFs describes a relationship between a listener and an audio source separated from the listener by a distance substantially equal to the first radius.

7. The method of claim 1 , wherein a distance from the source location to the center of the user's head is less than a radius of the user's head.

8. The method of claim 1 , wherein a distance from the source location to the center of the user's head is greater than a radius of the user's head.

9. The method of claim 1 , wherein a distance from the source location to the center of the user's head is substantially equal to a radius of the user's head.

10. The method of claim 1 , wherein the angle comprises one or more of an azimuth and an elevation angle.

11. The method of claim 1 , wherein the wearable head device comprises the one or more speakers.

12. The method of claim 1 , wherein the wearable head device does not comprise the one or more speakers.

13. The method of claim 1 , wherein the one or more speakers are associated with headphones worn by the user.

14. A system comprising:

a wearable head device;

one or more speakers; and

one or more processors configured to perform a method comprising:

identifying a source location corresponding to an audio signal;

determining an acoustic axis corresponding to the audio signal;

for each of a respective left and right ear of a user of the wearable head device:

determining an angle between the acoustic axis and the respective ear;

determining, of a virtual speaker array, a virtual speaker position collinear with the source location and a position of the respective ear, wherein the virtual speaker array comprises a plurality of virtual speaker positions, each virtual speaker position of the plurality of virtual speaker positions located on the surface of a sphere concentric with the user's head, the sphere having a first radius;

determining a head-related transfer function (HRTF) corresponding to the virtual speaker position and to the respective ear;

determining a source radiation filter based on the determined angle;

processing the audio signal to generate an output audio signal for the respective ear, wherein processing the audio signal comprises applying the HRTF and the source radiation filter to the audio signal;

attenuating the audio signal based on a distance between the source location and the respective ear wherein the distance is clamped at a minimum value; and

presenting the output audio signal to the respective ear of the user via the one or more speakers.

15. The system of claim 14 , wherein the source location is separated from a center of the user's head by a distance less than the first radius.

16. The system of claim 14 , wherein the source location is separated from a center of the user's head by a distance greater than the first radius.

17. The system of claim 14 , wherein the source location is separated from a center of the user's head by a distance equal to the first radius.

18. The system of claim 14 , wherein processing the audio signal further comprises applying an interaural time difference to the audio signal.

19. The system of claim 14 , wherein determining the HRTF corresponding to the virtual speaker position comprises selecting the HRTF from a plurality of HRTFs, wherein each HRTF of the plurality of HRTFs describes a relationship between a listener and an audio source separated from the listener by a distance substantially equal to the first radius.

20. The system of claim 14 , wherein a distance from the source location to the center of the user's head is less than a radius of the user's head.

21. The system of claim 14 , wherein a distance from the source location to the center of the user's head is greater than a radius of the user's head.

22. The system of claim 14 , wherein a distance from the source location to the center of the user's head is substantially equal to a radius of the user's head.

23. The system of claim 14 , wherein the angle comprises one or more of an azimuth and an elevation angle.

24. The system of claim 14 , wherein the wearable head device comprises the one or more speakers.

25. The system of claim 14 , wherein the wearable head device does not comprise the one or more speakers.

26. The system of claim 14 , wherein the one or more speakers are associated with headphones worn by the user.

27. A non-transitory computer-readable medium storing instructions, which when executed by one or more processors cause the one or more processors to perform a method of presenting an audio signal to a user of a wearable head device, the method comprising:

identifying a source location corresponding to the audio signal;

determining an acoustic axis corresponding to the audio signal;

for each of a respective left and right ear of the user:

determining an angle between the acoustic axis and the respective ear;

determining, of a virtual speaker array, a virtual speaker position collinear with the source location and a position of the respective ear, wherein the virtual speaker array comprises a plurality of virtual speaker positions, each virtual speaker position of the plurality of virtual speaker positions located on the surface of a sphere concentric with the user's head, the sphere having a first radius;

determining a head-related transfer function (HRTF) corresponding to the virtual speaker position and to the respective ear;

determining a source radiation filter based on the determined angle;

processing the audio signal to generate an output audio signal for the respective ear, wherein processing the audio signal comprises applying the HRTF and the source radiation filter to the audio signal;

attenuating the audio signal based on a distance between the source location and the respective ear wherein the distance is clamped at a minimum value; and

presenting the output audio signal to the respective ear of the user via one or more speakers associated with the wearable head device.

28. The non-transitory computer-readable medium of claim 27 , wherein the source location is separated from a center of the user's head by a distance less than the first radius.

29. The non-transitory computer-readable medium of claim 27 , wherein the source location is separated from a center of the user's head by a distance greater than the first radius.

30. The non-transitory computer-readable medium of claim 27 , wherein the source location is separated from a center of the user's head by a distance equal to the first radius.

31. The non-transitory computer-readable medium of claim 27 , wherein processing the audio signal further comprises applying an interaural time difference to the audio signal.

32. The non-transitory computer-readable medium of claim 27 , wherein determining the HRTF corresponding to the virtual speaker position comprises selecting the HRTF from a plurality of HRTFs, wherein each HRTF of the plurality of HRTFs describes a relationship between a listener and an audio source separated from the listener by a distance substantially equal to the first radius.

33. The non-transitory computer-readable medium of claim 27 , wherein a distance from the source location to the center of the user's head is less than a radius of the user's head.

34. The non-transitory computer-readable medium of claim 27 , wherein a distance from the source location to the center of the user's head is greater than a radius of the user's head.

35. The non-transitory computer-readable medium of claim 27 , wherein a distance from the source location to the center of the user's head is substantially equal to a radius of the user's head.

36. The non-transitory computer-readable medium of claim 27 , wherein the angle comprises one or more of an azimuth and an elevation angle.

37. The non-transitory computer-readable medium of claim 27 , wherein the wearable head device comprises the one or more speakers.

38. The non-transitory computer-readable medium of claim 27 , wherein the wearable head device does not comprise the one or more speakers.

39. The non-transitory computer-readable medium of claim 27 , wherein the one or more speakers are associated with headphones worn by the user.

Assignments (4)
SECURITY INTEREST Recorded Oct 15, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073109/0476 →
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
SECURITY INTEREST Recorded May 21, 2020
From: MAGIC LEAP, INC.; MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 052729/0791 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 6, 2020
From: AUDFRAY, REMI SAMUEL; JOT, JEAN-MARC; DICKER, SAMUEL CHARLES; HERTENSTEINER, MARK BRANDON; MATHEW, JUSTIN DAN; TAJIK, ANASTASIA ANDREYEVNA; LAMARTINA, NICHOLAS JOHN
To: MAGIC LEAP, INC.
Reel/Frame 051425/0441 →
Cited By (2)
US 12,317,062 US 12,342,158