IP Library Granted Patent US 11,770,671
Granted Patent B2
US 11,770,671 · App. 18/069,702 · Granted Sep 26, 2023

Spatial audio for interactive audio environments

Inventors: Remi Samuel Audfray (San Francisco, CA); Jean-Marc Jot (Aptos, CA); Samuel Charles Dicker (San Francisco, CA)
Assignee: Magic Leap, Inc.
H04S7/304G02B27/017G10K15/10H04R3/04H04R3/12H04R5/033H04R5/04H04S3/008H04S7/305G02B2027/0178H04R2499/15H04S2400/01H04S2400/13H04S2400/15H04S2420/11
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,770,671
App. No.
18/069,702
Granted
Sep 26, 2023
Kind
B2
Abstract

Systems and methods of presenting an output audio signal to a listener located at a first location in a virtual environment are disclosed. According to embodiments of a method, an input audio signal is received. A first intermediate audio signal corresponding to the input audio signal is determined, based on a location of the sound source in the virtual environment, and the first intermediate audio signal is associated with a first bus. A second intermediate audio signal is determined. The second intermediate audio signal corresponds to a reverberation of the input audio signal in the virtual environment. The second intermediate audio signal is determined based on a location of the sound source, and further based on an acoustic property of the virtual environment. The second intermediate audio signal is associated with a second bus. The output audio signal is presented to the listener via the first and second buses.

Claims (47)

1. A method comprising:

determining, based on a location of a sound source in a virtual environment, a first intermediate audio signal corresponding to an input audio signal;

associating the first intermediate audio signal with a first bus;

determining a distance between the location of the sound source and a location of a listener;

determining a second intermediate audio signal, the second intermediate audio signal associated with a reverberation of the input audio signal in the virtual environment;

associating the second intermediate audio signal with a second bus;

adjusting a gain of the second bus based on the distance; and

presenting, via the first bus and the second bus, an output audio signal to the listener.

2. The method of claim 1 , wherein the first intermediate audio signal is determined further based on an acoustic property of the virtual environment.

3. The method of claim 2 , wherein the acoustic property of the virtual environment is determined via one or more sensors associated with the listener.

4. The method of claim 3 , wherein the one or more sensors comprise one or more microphones.

5. The method of claim 2 , further comprising retrieving the acoustic property from a database.

6. The method of claim 2 , wherein the acoustic property is determined based on the location of the listener.

7. The method of claim 1 , wherein:

the output audio signal is presented to the listener via one or more speakers associated with a wearable head unit configured to be worn by the listener, and

the wearable head unit comprises a display configured to display to the listener, concurrently with the presentation of the output signal, a view of the virtual environment.

8. The method of claim 1 , wherein the location of the sound source is determined via one or more sensors associated with the listener.

9. The method of claim 1 , wherein the location of the listener is determined via one or more sensors associated with the listener.

10. The method of claim 9 , wherein:

the one or more sensors are associated with a wearable head unit configured to be worn by the listener, and

the output audio signal is presented to the listener via one or more speakers associated with the wearable head unit.

11. The method of claim 1 , wherein the distance between the location of the sound source and the location of the listener is determined via one or more sensors associated with the listener.

12. The method of claim 1 , wherein the determining the first intermediate audio signal comprises applying a filter to the input audio signal, the filter comprising one or more of a sound source directivity model, a distance model, and an orientation model.

13. The method of claim 1 , wherein the determining the first intermediate audio signal comprises applying a panning process to the input audio signal.

14. The method of claim 1 , wherein the gain is associated with one or more of a reverberation delay and a reverberation level.

15. The method of claim 14 , wherein the reverberation delay comprises one or more of a per-source delay and a global delay.

16. The method of claim 1 , wherein the gain is determined based on a frequency associated with the sound source.

17. The method of claim 1 , wherein the gain is determined based on the location of the sound source.

18. The method of claim 1 , wherein the gain comprises a normalized gain.

19. A device comprising:

one or more speakers;

one or more processors configured to perform a method comprising:

determining, based on a location of a sound source in a virtual environment, a first intermediate audio signal corresponding to an input audio signal;

associating the first intermediate audio signal with a first bus;

determining a distance between the location of the sound source and a location of a listener;

determining a second intermediate audio signal, the second intermediate audio signal associated with a reverberation of the input audio signal in the virtual environment;

associating the second intermediate audio signal with a second bus;

adjusting a gain of the second bus based on the distance; and

presenting, via the first bus, the second bus, and the speakers, an output audio signal to the listener.

20. A non-transitory computer-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform a method comprising:

determining, based on a location of a sound source in a virtual environment, a first intermediate audio signal corresponding to an input audio signal;

associating the first intermediate audio signal with a first bus;

determining a distance between the location of the sound source and a location of a listener;

determining a second intermediate audio signal, the second intermediate audio signal associated with a reverberation of the input audio signal in the virtual environment;

associating the second intermediate audio signal with a second bus;

adjusting a gain of the second bus based on the distance; and

presenting, via the first bus and the second bus, an output audio signal to the listener.

Assignments (3)
SECURITY INTEREST Recorded Oct 20, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073031/0206 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 25, 2023
From: AUDFRAY, REMI SAMUEL; JOT, JEAN-MARC; DICKER, SAMUEL CHARLES
To: MAGIC LEAP, INC.
Reel/Frame 064379/0971 →
SECURITY INTEREST Recorded Feb 7, 2023
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 062681/0065 →