Mixed reality spatial audio
A method of presenting an audio signal to a user of a mixed reality environment is disclosed. According to examples of the method, an audio event associated with the mixed reality environment is detected. The audio event is associated with a first audio signal. A location of the user with respect to the mixed reality environment is determined. An acoustic region associated with the location of the user is identified. A first acoustic parameter associated with the first acoustic region is determined. A transfer function is determined using the first acoustic parameter. The transfer function is applied to the first audio signal to produce a second audio signal, which is then presented to the user.
1. A method of presenting an audio signal to a user of a mixed reality environment, the method comprising:
detecting an audio event associated with the mixed reality environment, wherein the audio event is associated with a first audio signal;
determining a location of the user with respect to the mixed reality environment;
detecting, via a first sensor of a wearable device associated with the user, a spatial property of an environment of the user;
identifying, based on the detected spatial property, a first acoustic region;
determining a first acoustic parameter associated with the first acoustic region;
determining, using the first acoustic parameter, a transfer function;
applying the transfer function to the first audio signal to produce a second audio signal; and
presenting, to the user, the second audio signal.
2. The method of claim 1 , wherein the first audio signal comprises a waveform audio file.
3. The method of claim 1 , wherein the first audio signal comprises a live audio stream.
4. The method of claim 1 , wherein:
the wearable device comprises a display, and
the method comprises presenting the second audio signal to the user concurrently with presenting, via the display, a view of the mixed reality environment.
5. The method of claim 1 , wherein determining the first acoustic parameter comprises determining the first acoustic parameter based on input from the first sensor.
6. The method of claim 5 , wherein determining the first acoustic parameter further comprises identifying, based on the input from the first sensor, a geometric characteristic of the first acoustic region, and wherein the determining of the first acoustic parameter is based on the geometric characteristic.
7. The method of claim 5 , wherein determining the first acoustic parameter further comprises identifying, based on the input from the first sensor, a material associated with the first acoustic region, and wherein the determining of the first acoustic parameter is based on the material.
8. The method of claim 1 , wherein the wearable device comprises a microphone, wherein the microphone is located in the first acoustic region, and wherein the determining of the first acoustic parameter is based on a signal detected by the microphone.
9. The method of claim 1 , wherein the first acoustic parameter corresponds to a reverberation parameter.
10. The method of claim 1 , wherein the first acoustic parameter corresponds to a filtering parameter.
11. The method of claim 1 , further comprising:
detecting, via a second sensor of the wearable device, a second spatial property of the environment of the user;
identifying, based on the detected second spatial property, a second acoustic region, the second acoustic region acoustically coupled to the first acoustic region; and
determining a second acoustic parameter associated with the second acoustic region,
wherein determining the transfer function comprises determining the transfer function using both the first acoustic parameter and the second acoustic parameter.
12. The method of claim 11 , wherein the first sensor comprises the second sensor.
13. A system comprising:
a wearable device including:
a speaker; and
one or more sensors; and
one or more processors configured to perform a method comprising:
detecting an audio event associated with a mixed reality environment, wherein the audio event is associated with a first audio signal;
determining a location of the wearable device with respect to the mixed reality environment;
detecting, via a first sensor of the one or more sensors, a spatial property of an environment of the wearable device;
identifying, based on the detected spatial property, a first acoustic region;
determining a first acoustic parameter associated with the first acoustic region;
determining, using the first acoustic parameter, a transfer function;
applying the transfer function to the first audio signal to produce a second audio signal; and
presenting, via the speaker, the second audio signal.
14. The system of claim 13 , wherein the first audio signal comprises a waveform audio file.
15. The system of claim 13 , wherein the first audio signal comprises a live audio stream.
16. The system of claim 13 , wherein:
the wearable device further includes a display, and
the method comprises presenting the second audio signal via the speaker concurrently with presenting, via the display, a view of the mixed reality environment.
17. The system of claim 16 , wherein determining the first acoustic parameter comprises determining the first acoustic parameter based on input from the first sensor.
18. The system of claim 17 , wherein determining the first acoustic parameter further comprises identifying, based on the input from the first sensor, a geometric characteristic of the first acoustic region, and wherein the determining of the first acoustic parameter is based on the geometric characteristic.
19. The system of claim 17 , wherein determining the first acoustic parameter further comprises identifying, based on the input from the first sensor, a material associated with the first acoustic region, and wherein the determining of the first acoustic parameter is based on the material.
20. The system of claim 13 , wherein the one or more sensors comprises a microphone and wherein the determining of the first acoustic parameter is based on a signal detected by the microphone.
21. The system of claim 13 , wherein the first acoustic parameter corresponds to a reverberation parameter.
22. The system of claim 13 , wherein the first acoustic parameter corresponds to a filtering parameter.
23. The system of claim 13 , wherein the method further comprises:
detecting, via a second sensor of the one or more sensors, a second spatial property of the environment of the wearable device;
identifying, based on the detected second spatial property, a second acoustic region, the second acoustic region acoustically coupled to the first acoustic region; and
determining a second acoustic parameter associated with the second acoustic region,
wherein the determining the transfer function comprises determining the transfer function using both the first acoustic parameter and the second acoustic parameter.
24. The system of claim 23 , wherein the first sensor comprises the second sensor.
25. A non-transitory computer-readable storage medium comprising instructions, which when executed by one or more processors, cause the one or more processors to execute a method comprising:
detecting an audio event associated with a mixed reality environment, wherein the audio event is associated with a first audio signal;
determining a location of a user with respect to the mixed reality environment;
detecting, via a first sensor of a wearable device associated with the user, a spatial property of an environment of the user;
identifying, based on the detected spatial property, a first acoustic region;
determining a first acoustic parameter associated with the first acoustic region;
determining, using the first acoustic parameter, a transfer function;
applying the transfer function to the first audio signal to produce a second audio signal; and
presenting, to the user, the second audio signal.