Audio capture with multiple devices
In one implementation, a method of visualizing a combined audio pick-up pattern is performed at a first device in a physical environment, the first device including a display, one or more processors, and non-transitory memory. The method includes determining a first audio pick-up pattern of the first device. The method includes determining one or more second audio pick-up patterns of a respective one or more second devices. The method includes determining a combined audio pick-up pattern of the first device and the one or more second devices based on the first audio pick-up pattern and the one or more second audio pick-up patterns. The method includes displaying, on the display, a representation of the combined audio pick-up pattern. In one implementation, a method of determining an audio emission pattern is performed at a first device at a first location, the first device having a microphone, one or more processors, and non-transitory memory. The method includes obtaining, via the microphone, first audio of a sound source. The method includes receiving, from one or more second devices, one or more second audio of the sound source. The method includes determining one or more second locations of the one or more second devices. The method includes determining an audio emission pattern of the sound source based on the first audio data, the one or more second audio data, and the one or more second locations, wherein the audio emission pattern of the sound source indicates a sound level at various locations relative to the sound source.
1. A method comprising:
at a first device at a first location, the first device having a microphone, one or more processors, and non-transitory memory;
obtaining, via the microphone, first audio of a sound source;
receiving, from one or more second devices, one or more second audio of the sound source;
determining one or more second locations of the one or more second devices; and
determining an audio emission pattern of the sound source based on the first audio, the one or more second audio, and the one or more second locations, wherein the audio emission pattern of the sound source indicates a sound level at various locations relative to the sound source.
2. The method of claim 1 , wherein determining the second location of a particular second device includes detecting the particular second device in one or more images of a physical environment.
3. The method of claim 2 , wherein determining the second location of the particular second device includes identifying the particular second device based on data encoded in the one or more images of the physical environment.
4. The method of claim 1 , wherein determining the audio emission pattern of the sound source includes generating combined audio based on the first audio and the one or more second audio.
5. The method of claim 4 , wherein generating the combined audio includes synchronizing the first audio and the one or more second audio.
6. The method of claim 4 , wherein generating the combined audio includes filtering direct audio from reverberation audio in at least one of the first audio and one or more second audio.
7. The method of claim 4 , wherein generating the combined audio includes performing noise reduction in at least one of the first audio and one or more second audio.
8. The method of claim 4 , wherein determining the audio emission pattern includes determining a base volume of the sound source at each of a plurality of times and each of a plurality of frequencies based on a volume of the combined audio at the time and frequency.
9. The method of claim 8 , wherein determining the audio emission pattern includes, for the first audio associated with the first device and each of the one or more second audio respectively associated with the one or more second devices, determining a directional volume of the sound source at each of the plurality of times and each of the plurality of frequencies based on a volume of the audio at the time, a pose of the respective device, and an audio pick-up pattern of the respective device.
10. The method of claim 9 , wherein determining the audio emission pattern includes, for the first audio and each of the one or more second audio, determining a normalized directional volume of sound at each of the plurality of times and each of the plurality of frequencies by dividing the directional volume of sound at the time and frequency by the base volume of sound at the time and frequency.
11. The method of claim 1 , wherein the audio emission pattern of the sound source indicates, at a particular location relative to the sound source, a sound level at various frequencies.
12. The method of claim 1 , wherein the audio emission pattern of the sound source indicates, at a particular location relative to the sound source, a sound level at various times.
13. The method of claim 1 , wherein determining the audio emission pattern of the sound source includes determining a sound level at a third location, different from the first location and the one or more second locations, based on at least two of a sound level at the first location and one or more sound levels at the one or more second locations.
14. The method of claim 1 , further comprising storing the audio emission pattern of the sound source in association with combined audio based on the first audio and the one or more second audio.
15. A device at a first location comprising:
a microphone;
non-transitory memory; and
one or more processors to:
obtain, via the microphone, first audio of a sound source;
receive, from one or more second devices, one or more second audio of the sound source;
determine one or more second locations of the one or more second devices; and
determine an audio emission pattern of the sound source based on the first audio, the one or more second audio, and the one or more second locations, wherein the audio emission pattern of the sound source indicates a sound level at various locations relative to the sound source.
16. The device of claim 15 , wherein the one or more processors are to determine the audio emission pattern of the sound source by generating combined audio based on the first audio and the one or more second audio.
17. The device of claim 15 , wherein the audio emission pattern of the sound source indicates, at a particular location relative to the sound source, a sound level at various times and frequencies.
18. The device of claim 15 , wherein the one or more processors are to determine the audio emission pattern of the sound source by determining a sound level at a third location, different from the first location and the one or more second locations, based on at least two of a sound level at the first location and one or more sound levels at the one or more second locations.
19. The device of claim 15 , wherein the one or more processors are further to store the audio emission pattern of the sound source in association with combined audio based on the first audio and the one or more second audio.
20. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device at a first location including a microphone, cause the device to:
obtain, via the microphone, first audio of a sound source;
receive, from one or more second devices, one or more second audio of the sound source;
determine one or more second locations of the one or more second devices; and
determine an audio emission pattern of the sound source based on the first audio, the one or more second audio, and the one or more second locations, wherein the audio emission pattern of the sound source indicates a sound level at various locations relative to the sound source.