IP Library Granted Patent US 11,477,510
Granted Patent B2
US 11,477,510 · App. 16/970,331 · Granted Oct 18, 2022

Mixed reality virtual reverberation

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,477,510
App. No.
16/970,331
Filed
Aug 14, 2020
Granted
Oct 18, 2022
Kind
B2
Art Unit
2426
USPC
348/552
Abstract

A method of presenting an audio signal to a user of a mixed reality environment is disclosed, the method comprising the steps of detecting a first audio signal in the mixed reality environment, where the first audio signal is a real audio signal; identifying a virtual object intersected by the first audio signal in the mixed reality environment; identifying a listener coordinate associated with the user; determining, using the virtual object and the listener coordinate, a transfer function; applying the transfer function to the first audio signal to produce a second audio signal; and presenting, to the user, the second audio signal.

Claims (47)

1. A method of presenting an audio signal to a user of a mixed reality environment, the method comprising:

detecting a first audio signal in the mixed reality environment, where the first audio signal is a real audio signal;

identifying a virtual object intersected by the first audio signal in the mixed reality environment;

identifying, via one or more inertial measurement units (IMUs) of a wearable head device, a position of the user in the mixed reality environment, wherein the one or more IMUs are configured for six-degree-of-freedom tracking of a head pose of the user;

identifying a listener coordinate associated with the user based on the position of the user;

determining, using the virtual object and the listener coordinate, a transfer function;

applying the transfer function to the first audio signal to produce a second audio signal; and

presenting, via a speaker of the wearable head device, to the user, the second audio signal.

2. The method of claim 1 , wherein detecting the first audio signal comprises detecting a voice signal generated by the user.

3. The method of claim 1 , further comprising identifying, via the one or more inertial measurement units, an orientation of the user in the mixed reality environment, wherein the transfer function is determined using the orientation of the user.

4. The method of claim 1 , wherein the first audio signal is a user-generated signal.

5. The method of claim 1 , further comprising determining an angle at which the first audio signal intersects the virtual object, wherein the transfer function is determined using the angle.

6. The method of claim 1 , wherein the first audio signal is associated with a direction in the mixed reality environment, and the transfer function is determined using the direction.

7. The method of claim 1 , wherein:

the virtual object is associated with a material, and

the transfer function is determined using the material.

8. The method of claim 1 , wherein the transfer function comprises a function of one or more of time, frequency, gain, and phase.

9. The method of claim 1 , wherein the transfer function comprises a head-related transfer function.

10. The method of claim 1 , further comprising determining an origin and direction of the second audio signal in the mixed reality environment based on the intersection between the first audio signal and the virtual object.

11. The method of claim 1 , further comprising performing feedback cancellation on the first audio signal comprising computing a difference between the first audio signal and an audio signal previously presented to the user.

12. The method of claim 1 , further comprising identifying a direction of the first audio signal comprising measuring a time delay between when the first signal reaches each of two microphones.

13. A system comprising:

a wearable head device including:

a display for displaying a mixed reality environment to a user, the display comprising a transmissive eyepiece through which a real environment is visible;

a microphone;

a speaker; and

one or more inertial measurement unit (IMUs); and

one or more processors configured to perform:

detecting, via the microphone, a first audio signal in the mixed reality environment, where the first audio signal is a real audio signal;

identifying a virtual object intersected by the first audio signal in the mixed reality environment;

presenting, to the user via the display, the virtual object;

identifying, via the one or more IMUs, a position of the wearable head device in the mixed reality environment, wherein the one or more IMUs are configured for six-degree-of-freedom tracking of a head pose of the user;

identifying a listener coordinate based on the position of the wearable head device;

determining, using the virtual object and the listener coordinate, a transfer function;

applying the transfer function to the first audio signal to produce a second audio signal; and

presenting, to the user via the speaker, the second audio signal.

14. The system of claim 13 , wherein detecting, via the microphone, the first audio signal comprises detecting, via the microphone, a voice signal generated by the user.

15. The system of claim 13 , wherein the one or more processors are further configured to perform identifying, via the one or more inertial measurement units, an orientation of the wearable head device in the mixed reality environment, and wherein the transfer function is determined using the orientation of the wearable head device.

16. The system of claim 13 , wherein the one or more processors are further configured to perform identifying, via the inertial measurement unit, an orientation of the user in the mixed reality environment, and wherein the transfer function is further determined using the orientation of the user.

17. The system of claim 13 , wherein the first audio signal is a user-generated signal.

18. The system of claim 13 , wherein the one or more processors are further configured to perform determining an angle at which the first audio signal intersects the virtual object, and wherein the transfer function is determined using the angle.

19. The system of claim 13 , wherein the first audio signal is associated with a direction in the mixed reality environment, and the transfer function is determined using the direction.

20. The system of claim 13 , wherein:

the virtual object is associated with a material, and

the transfer function is determined using the material.

21. The system of claim 13 , wherein the one or more processors are further configured to perform feedback cancellation on the first audio signal comprising computing a difference between the first audio signal and an audio signal previously presented to the user.

22. The system of claim 13 , further comprising a second microphone, wherein the one or more processors are further configured to perform identifying a direction of the first audio signal comprising measuring a time delay between when the first signal reaches the first and second microphones.