IP Library Granted Patent US 11,477,592
Granted Patent B2
US 11,477,592 · App. 16/987,079 · Granted Oct 18, 2022

Methods and systems for audio signal filtering

Inventors: Remi Samuel Audfray (San Francisco, CA); Jean-Marc Jot (Aptos, CA); Samuel Charles Dicker (San Francisco, CA)
Assignee: Magic Leap, Inc.
H04S1/007H04S1/005H04S7/304H04R5/033H04S2400/11H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,477,592
App. No.
16/987,079
Granted
Oct 18, 2022
Kind
B2
Abstract

Systems and methods for rendering audio signals are disclosed. In some embodiments, a method may receive an input signal including a first portion and the second portion. A first processing stage comprising a first filter is applied to the first portion to generate a first filtered signal. A second processing stage comprising a second filter is applied to the first portion to generate a second filtered signal. A third processing stage comprising a third filter is applied to the second portion to generate a third filtered signal. A fourth processing stage comprising a fourth filter is applied to the second portion to generate a fourth filtered signal. A first output signal is determined based on a sum of the first filtered signal and the third filtered signal. A second output signal is determined based on a sum of the second filtered signal and the fourth filtered signal. The first output signal is presented to a first ear of a user of a virtual environment, and the second output signal is presented to the second ear of the user. The first portion of the input signal corresponds to a first location in the virtual environment, and the second portion of the input signal corresponds to a second location in the virtual environment.

Claims (74)

1. A method comprising:

accepting, at an amplitude panning stage, an audio input signal;

processing the audio input signal to generate a first processed signal and a second processed signal, wherein the processing comprises:

trimming at least one of a first filter and a second filter,

applying the first filter to the audio input signal, and

applying the second filter to the audio input signal;

adjusting, with respect to a first virtual speaker, a gain of the first processed signal to generate a first virtual speaker output;

adjusting, with respect to a second virtual speaker, a gain of the second processed signal to generate a second virtual speaker output, wherein the first virtual speaker output and the second virtual speaker output are adjusted to achieve a signal balance across the first virtual speaker and the second virtual speaker;

determining a virtual speaker array output based on a sum of the first virtual speaker output and the second virtual speaker output; and

sending the virtual speaker array output to a first speaker.

2. The method of claim 1 , wherein the audio input signal comprises a first portion and a second portion.

3. The method of claim 2 , wherein processing the audio input signal comprises:

applying a first filter to the first portion of the audio input signal to generate a first filtered signal;

applying a second filter to the first portion of the audio input signal to generate a second filtered signal;

applying a third filter to the second portion of the audio input signal to generate a third filtered signal;

applying a fourth filter to the second portion of the audio input signal to generate a fourth filtered signal;

determining the first processed signal based on a sum of the first filtered signal and the third filtered signal; and

determining the second processed signal based on a sum of the second filtered signal and the fourth filtered signal.

4. The method of claim 3 , wherein the first portion of the audio input signal corresponds to a first location of an environment and the second portion of the audio input signal corresponds to the first location of the environment.

5. The method of claim 1 , wherein trimming at least one of the first filter and second filter comprises:

setting a threshold level;

removing portions of the audio input signal located before the audio input signal crosses the threshold level.

6. The method of claim 1 , wherein trimming at least one of the first filter and second filter comprises:

identifying a peak in the audio input signal; and

applying a predetermined window around the identified peak; and

removing portions of the signal outside of the predetermined window.

7. The method of claim 1 , wherein aligning the first filter and the second filter comprises generating a minimum phase version of each of the first filter and the second filter.

8. A system comprising:

a wearable head device including a first speaker and a second speaker; and

one or more processors configured to perform a method comprising:

accepting, at an amplitude panning stage, an audio input signal;

processing the audio input signal to generate a first processed signal and a second processed signal, wherein the processing comprises:

trimming at least one of a first filter and a second filter,

applying the first filter to the audio input signal, and

applying the second filter to the audio input signal;

adjusting, with respect to a first virtual speaker, a gain of the first processed signal to generate a first virtual speaker output;

adjusting, with respect to a second virtual speaker, a gain of the second processed signal to generate a second virtual speaker output, wherein the first virtual speaker output and the second virtual speaker output are adjusted to achieve a signal balance across the first virtual speaker and the second virtual speaker;

determining a virtual speaker array output based on a sum of the first virtual speaker output and the second virtual speaker output; and

sending the virtual speaker array output to at least one of the first speaker and second speaker.

9. The system of claim 8 , wherein the audio input signal comprises a first portion and a second portion.

10. The system of claim 9 , wherein processing the audio input signal comprises:

applying a first filter to the first portion of the audio input signal to generate a first filtered signal;

applying a second filter to the first portion of the audio input signal to generate a second filtered signal;

applying a third filter to the second portion of the audio input signal to generate a third filtered signal;

applying a fourth filter to the second portion of the audio input signal to generate a fourth filtered signal;

determining the first processed signal based on a sum of the first filtered signal and the third filtered signal; and

determining the second processed signal based on a sum of the second filtered signal and the fourth filtered signal.

11. The system of claim 10 , wherein the first portion of the audio input signal corresponds to a first location of an environment and the second portion of the audio input signal corresponds to the first location of the environment.

12. The system of claim 8 , further comprising one or more microphones, wherein the one or more processors are configured to receive the audio input signal via the one or more microphones.

13. A system comprising:

a wearable head device including a first speaker and a second speaker;

one or more sensors;

a display configured to present a view of a virtual environment; and

one or more processors configured to perform a method comprising:

accepting, at an amplitude panning stage, an audio input signal;

processing the audio input signal to generate a first processed signal and a second processed signal, wherein the processing comprises:

trimming at least one of a first filter and a second filter,

applying the first filter to the audio input signal, and

applying the second filter to the audio input signal;

adjusting, with respect to a first virtual speaker, a gain of the first processed signal to generate a first virtual speaker output;

adjusting, with respect to a second virtual speaker, a gain of the second processed signal to generate a second virtual speaker output, wherein the first virtual speaker output and the second virtual speaker output are adjusted to achieve a signal balance across the first virtual speaker and second virtual speaker;

determining a virtual speaker array output based on a sum of the first virtual speaker output and the second virtual speaker output; and

presenting, via at least one of the first speaker and second speaker, an audio output based on the virtual speaker array output.

14. The system of claim 13 , wherein the audio input signal comprises a first portion and a second portion.

15. The system of claim 14 , wherein processing the audio input signal comprises:

applying a first filter to the first portion of the audio input signal to generate a first filtered signal;

applying a second filter to the first portion of the audio input signal to generate a second filtered signal;

applying a third filter to the second portion of the audio input signal to generate a third filtered signal;

applying a fourth filter to the second portion of the audio input signal to generate a fourth filtered signal;

determining the first processed signal based on a sum of the first filtered signal and the third filtered signal; and

determining the second processed signal based on a sum of the second filtered signal and the fourth filtered signal.

16. The system of claim 15 , wherein the first portion of the audio input signal corresponds to a first location of an environment and the second portion of the audio input signal corresponds to the first location of the environment.

17. The system of claim 13 , further comprising one or more microphones, wherein the one or more processors are configured to receive the audio input signal via the one or more microphones.

18. The system of claim 13 , wherein the one or more sensors comprises one or more of a camera, a LIDAR sensor, a sonar sensor, an orientation sensor, and a GPS sensor.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 25, 2023
From: AUDFRAY, REMI SAMUEL; JOT, JEAN-MARC; DICKER, SAMUEL CHARLES
To: MAGIC LEAP, INC.
Reel/Frame 064379/0722 →
SECURITY INTEREST Recorded Feb 7, 2023
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 062681/0065 →
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →