IP Library Granted Patent US 10,887,720
Granted Patent B2
US 10,887,720 · App. 16/593,944 · Granted Jan 5, 2021

Emphasis for audio spatialization

Inventor: Samuel Charles Dicker (San Francisco, CA)
Assignee: Magic Leap, Inc.
H04S7/304H04R3/04H04R5/033H04R5/04H04S3/008H04S7/307H04S2400/01H04S2400/11H04S2400/13H04S2420/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,887,720
App. No.
16/593,944
Granted
Jan 5, 2021
Kind
B2
Abstract

Examples of the disclosure describe systems and methods for presenting an audio signal to a user of a wearable head device. According to an example method, a first input audio signal is received. The first input audio signal is processed to generate a first output audio signal. The first output audio signal is presented via one or more speakers associated with the wearable head device. Processing the first input audio signal comprises applying a pre-emphasis filter to the first input audio signal; adjusting a gain of the first input audio signal; and applying a de-emphasis filter to the first audio signal. Applying the pre-emphasis filter to the first input audio signal comprises attenuating a low frequency component of the first input audio signal. Applying the de-emphasis filter to the first input audio signal comprises attenuating a high frequency component of the first input audio signal.

Claims (128)

1. A method of presenting an audio signal to a user of a wearable head device, the method comprising:

receiving a first input audio signal, the first input audio signal associated with a virtual environment presented on a display of the wearable head device;

processing the first input audio signal to generate a first output audio signal, the first output audio signal associated with the virtual environment, wherein processing the first input audio signal comprises:

applying a pre-emphasis filter to the first input audio signal;

adjusting a gain of the first input audio signal; and

applying a de-emphasis filter to the first input audio signal; and

presenting the first output audio signal via one or more speakers associated with the wearable head device,

wherein:

applying the pre-emphasis filter to the first input audio signal comprises attenuating a low frequency component of the first input audio signal, and

applying the de-emphasis filter to the first input audio signal comprises attenuating a high frequency component of the first input audio signal.

2. The method of claim 1 , wherein the pre-emphasis filter comprises a first derivative filter.

3. The method of claim 2 , wherein the first derivative filter has a per-octave roll off of approximately six decibels.

4. The method of claim 1 , wherein applying the de-emphasis filter to the first input audio signal further comprises maintaining or increasing an amplitude of a low frequency component of the first input audio signal.

5. The method of claim 1 , wherein the de-emphasis filter comprises an integrator filter.

6. The method of claim 1 , wherein the de-emphasis filter comprises a leaky integrator with a per-octave boost of approximately six decibels.

7. The method of claim 1 , wherein the de-emphasis filter comprises a DC blocking filter.

8. The method of claim 1 , further comprising receiving a second input audio signal,

wherein:

processing the first input audio signal to generate the first output audio signal further comprises mixing, via a mixer, the first input audio signal with the second input audio signal.

9. The method of claim 1 , wherein presenting the first output audio signal via one or more speakers of the wearable head device comprises:

applying a first head-related transfer function (HRTF) to the first output audio signal;

presenting the output of the first HRTF to a left speaker of the one or more speakers of the wearable head device;

applying a second HRTF to the first output audio signal; and

presenting the output of the second HRTF to a right speaker of the one or more speakers of the wearable head device.

10. The method of claim 1 , wherein processing the first input audio signal to generate the first output audio signal further comprises:

applying an output of the pre-emphasis filter to one or more filters;

panning a first output of the one or more filters to generate a first panned signal, a second panned signal, a third panned signal, and a fourth panned signal;

applying the first panned signal to a left bus;

applying the second panned signal to a right bus;

applying the third panned signal to a standard bus;

applying the fourth panned signal to a diffuse bus; and

applying the left bus, the right bus, the standard bus, and the diffuse bus as input to a virtualizer,

wherein applying the de-emphasis filter to the first audio signal comprises applying the de-emphasis filter to the output of the virtualizer.

11. The method of claim 10 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a pre-delay to the first panned signal and the second panned signal.

12. The method of claim 10 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a decorrelation filter to the diffuse bus.

13. The method of claim 10 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a second output of the one or more filters as input to a clustered reflections module, and applying the output of the clustered reflections module to the standard bus.

14. The method of claim 10 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a second output of the one or more filters as input to a reverb module, and applying the output of the reverb module to the standard bus.

15. The method of claim 10 , wherein the one or more filters comprises a distance filter.

16. The method of claim 10 , wherein the one or more filters comprises an air absorption filter.

17. The method of claim 10 , wherein the one or more filters comprises a source directivity filter.

18. The method of claim 10 , wherein the one or more filters comprises an occlusion filter.

19. The method of claim 10 , wherein the one or more filters comprises an obstruction filter.

20. A system comprising:

a wearable head device;

one or more speakers;

one or more processors configured to execute a method comprising:

receiving a first input audio signal, the first input audio signal associated with a virtual environment presented on a display of the wearable head device;

processing the first input audio signal to generate a first output audio signal, the first output audio signal associated with the virtual environment, wherein processing the first input audio signal comprises:

applying a pre-emphasis filter to the first input audio signal;

adjusting a gain of the first input audio signal; and

applying a de-emphasis filter to the first input audio signal; and

presenting the first output audio signal via the one or more speakers,

wherein:

applying the pre-emphasis filter to the first input audio signal comprises attenuating a low frequency component of the first input audio signal, and

applying the de-emphasis filter to the first input audio signal comprises attenuating a high frequency component of the first input audio signal.

21. The system of claim 20 , wherein the pre-emphasis filter comprises a first derivative filter.

22. The system of claim 21 , wherein the first derivative filter has a per-octave roll off of approximately six decibels.

23. The system of claim 20 , wherein applying the de-emphasis filter to the first input audio signal further comprises maintaining or increasing an amplitude of a low frequency component of the first input audio signal.

24. The system of claim 20 , wherein the de-emphasis filter comprises an integrator filter.

25. The system of claim 20 , wherein the de-emphasis filter comprises a leaky integrator with a per-octave boost of approximately six decibels.

26. The system of claim 20 , wherein the de-emphasis filter comprises a DC blocking filter.

27. The system of claim 20 , the method further comprising receiving a second input audio signal, wherein:

processing the first input audio signal to generate the first output audio signal further comprises mixing, via a mixer, the first input audio signal with the second input audio signal.

28. The system of claim 20 , wherein presenting the first output audio signal via one or more speakers of the wearable head device comprises:

applying a first head-related transfer function (HRTF) to the first output audio signal;

presenting the output of the first HRTF to a left speaker of the one or more speakers of the wearable head device;

applying a second HRTF to the first output audio signal; and

presenting the output of the second HRTF to a right speaker of the one or more speakers of the wearable head device.

29. The system of claim 20 , wherein processing the first input audio signal to generate the first output audio signal further comprises:

applying an output of the pre-emphasis filter to one or more filters;

panning a first output of the one or more filters to generate a first panned signal, a second panned signal, a third panned signal, and a fourth panned signal;

applying the first panned signal to a left bus;

applying the second panned signal to a right bus;

applying the third panned signal to a standard bus;

applying the fourth panned signal to a diffuse bus; and

applying the left bus, the right bus, the standard bus, and the diffuse bus as input to a virtualizer,

wherein applying the de-emphasis filter to the first audio signal comprises applying the de-emphasis filter to the output of the virtualizer.

30. The system of claim 29 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a pre-delay to the first panned signal and the second panned signal.

31. The system of claim 29 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a decorrelation filter to the diffuse bus.

32. The system of claim 29 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a second output of the one or more filters as input to a clustered reflections module, and applying the output of the clustered reflections module to the standard bus.

33. The system of claim 29 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a second output of the one or more filters as input to a reverb module, and applying the output of the reverb module to the standard bus.

34. The system of claim 29 , wherein the one or more filters comprises a distance filter.

35. The system of claim 29 , wherein the one or more filters comprises an air absorption filter.

36. The system of claim 29 , wherein the one or more filters comprises a source directivity filter.

37. The system of claim 29 , wherein the one or more filters comprises an occlusion filter.

38. The system of claim 29 , wherein the one or more filters comprises an obstruction filter.

39. A non-transitory computer-readable medium storing instructions, which when executed by one or more processors cause the one or more processors to perform a method of presenting an audio signal to a user of a wearable head device, the method comprising:

receiving a first input audio signal, the first input audio signal associated with a virtual environment presented on a display of the wearable head device;

processing the first input audio signal to generate a first output audio signal, the first output audio signal associated with the virtual environment, wherein processing the first input audio signal comprises:

applying a pre-emphasis filter to the first input audio signal;

adjusting a gain of the first input audio signal; and

applying a de-emphasis filter to the first input audio signal; and

presenting the first output audio signal via one or more speakers associated with the wearable head device,

wherein:

applying the pre-emphasis filter to the first input audio signal comprises attenuating a low frequency component of the first input audio signal, and

applying the de-emphasis filter to the first input audio signal comprises attenuating a high frequency component of the first input audio signal.

40. The non-transitory computer-readable medium of claim 39 , wherein the pre-emphasis filter comprises a first derivative filter.

41. The non-transitory computer-readable medium of claim 40 , wherein the first derivative filter has a per-octave roll off of approximately six decibels.

42. The non-transitory computer-readable medium of claim 39 , wherein applying the de-emphasis filter to the first input audio signal further comprises maintaining or increasing an amplitude of a low frequency component of the first input audio signal.

43. The non-transitory computer-readable medium of claim 39 , wherein the de-emphasis filter comprises an integrator filter.

44. The non-transitory computer-readable medium of claim 39 , wherein the de-emphasis filter comprises a leaky integrator with a per-octave boost of approximately six decibels.

45. The non-transitory computer-readable medium of claim 39 , wherein the de-emphasis filter comprises a DC blocking filter.

46. The non-transitory computer-readable medium of claim 39 , the method further comprising receiving a second input audio signal,

wherein:

processing the first input audio signal to generate the first output audio signal further comprises mixing, via a mixer, the first input audio signal with the second input audio signal.

47. The non-transitory computer-readable medium of claim 39 , wherein presenting the first output audio signal via one or more speakers of the wearable head device comprises:

applying a first head-related transfer function (HRTF) to the first output audio signal;

presenting the output of the first HRTF to a left speaker of the one or more speakers of the wearable head device;

applying a second HRTF to the first output audio signal; and

presenting the output of the second HRTF to a right speaker of the one or more speakers of the wearable head device.

48. The non-transitory computer-readable medium of claim 39 , wherein processing the first input audio signal to generate the first output audio signal further comprises:

applying an output of the pre-emphasis filter to one or more filters;

panning a first output of the one or more filters to generate a first panned signal, a second panned signal, a third panned signal, and a fourth panned signal;

applying the first panned signal to a left bus;

applying the second panned signal to a right bus;

applying the third panned signal to a standard bus;

applying the fourth panned signal to a diffuse bus; and

applying the left bus, the right bus, the standard bus, and the diffuse bus as input to a virtualizer,

wherein applying the de-emphasis filter to the first audio signal comprises applying the de-emphasis filter to the output of the virtualizer.

49. The non-transitory computer-readable medium of claim 48 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a pre-delay to the first panned signal and the second panned signal.

50. The non-transitory computer-readable medium of claim 48 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a decorrelation filter to the diffuse bus.

51. The non-transitory computer-readable medium of claim 48 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a second output of the one or more filters as input to a clustered reflections module, and applying the output of the clustered reflections module to the standard bus.

52. The non-transitory computer-readable medium of claim 48 , wherein processing the first input audio signal to generate the first output audio signal further comprises applying a second output of the one or more filters as input to a reverb module, and applying the output of the reverb module to the standard bus.

53. The non-transitory computer-readable medium of claim 48 , wherein the one or more filters comprises a distance filter.

54. The non-transitory computer-readable medium of claim 48 , wherein the one or more filters comprises an air absorption filter.

55. The non-transitory computer-readable medium of claim 48 , wherein the one or more filters comprises a source directivity filter.

56. The non-transitory computer-readable medium of claim 48 , wherein the one or more filters comprises an occlusion filter.

57. The non-transitory computer-readable medium of claim 48 , wherein the one or more filters comprises an obstruction filter.

Assignments (3)
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
SECURITY INTEREST Recorded May 21, 2020
From: MAGIC LEAP, INC.; MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 052729/0791 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 6, 2020
From: DICKER, SAMUEL CHARLES
To: MAGIC LEAP, INC.
Reel/Frame 051425/0871 →