Audio zoom
A device includes one or more processors configured to execute instructions to determine a first phase based on a first audio signal of first audio signals and to determine a second phase based on a second audio signal of second audio signals. The one or more processors are also configured to execute the instructions to apply spatial filtering to selected audio signals of the first audio signals and the second audio signals to generate an enhanced audio signal. The one or more processors are further configured to execute the instructions to generate a first output signal including combining a magnitude of the enhanced audio signal with the first phase and to generate a second output signal including combining the magnitude of the enhanced audio signal with the second phase. The first output signal and the second output signal correspond to an audio zoomed signal.
1. A device comprising:
a memory configured to store instructions; and
one or more processors configured to execute the instructions to:
determine a first phase based on a first audio signal of first audio signals;
determine a second phase based on a second audio signal of second audio signals;
apply spatial filtering to selected audio signals of the first audio signals and the second audio signals, wherein the spatial filtering is applied to a first subset of the selected audio signals to generate a first enhanced audio signal, and wherein the spatial filtering is applied to a second subset of the selected audio signals to generate a second enhanced audio signal;
select one of the first enhanced audio signal or the second enhanced audio signal as an enhanced audio signal;
generate a first output signal including combining a magnitude of the enhanced audio signal with the first phase; and
generate a second output signal including combining the magnitude of the enhanced audio signal with the second phase, wherein the first output signal and the second output signal correspond to an audio zoomed signal.
2. The device of claim 1 , wherein the one or more processors are further configured to:
receive the first audio signals from a first plurality of microphones mounted externally to a first earpiece of a headset; and
receive the second audio signals from a second plurality of microphones mounted externally to a second earpiece of the headset.
3. The device of claim 2 , wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, a configuration of the first plurality of microphones and the second plurality of microphones, or a combination thereof.
4. The device of claim 3 , wherein the one or more processors are configured to determine the zoom direction, the zoom depth, or both, based on a tap detected via a touch sensor of the headset.
5. The device of claim 3 , wherein the one or more processors are configured to determine the zoom direction, the zoom depth, or both, based on a movement of the headset.
6. The device of claim 1 , wherein the one or more processors are integrated into a headset.
7. The device of claim 1 , wherein the one or more processors are further configured to:
provide the first output signal to a first speaker of a first earpiece of a headset; and
provide the second output signal to a second speaker of a second earpiece of the headset.
8. The device of claim 1 , wherein the one or more processors are further configured to decode audio data of a playback file to generate the first audio signals and the second audio signals.
9. The device of claim 8 , wherein the audio data includes position information indicating positions of sources of each of the first audio signals and the second audio signals, and wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, the position information, or a combination thereof.
10. The device of claim 8 , wherein the audio data includes a multi-channel audio representation of one or more audio sources, and wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, the multi-channel audio representation, or a combination thereof.
11. The device of claim 10 , wherein the multi-channel audio representation corresponds to ambisonics data.
12. The device of claim 1 , further comprising a modem coupled to the one or more processors, the modem configured to provide audio data to the one or more processors based on received streaming data, wherein the one or more processors are configured to decode the audio data to generate the first audio signals and the second audio signals.
13. The device of claim 1 , wherein the one or more processors are integrated into a vehicle, and wherein the one or more processors are configured to:
apply the spatial filtering based on a first location of a first occupant of the vehicle; and
provide the first output signal and the second output signal to a first speaker and a second speaker, respectively, to play out the audio zoomed signal to a second occupant of the vehicle.
14. The device of claim 13 , wherein the one or more processors are configured to:
position a movable mounting structure based on the first location of the first occupant; and
receive the first audio signals and the second audio signals from a plurality of microphones mounted on the movable mounting structure.
15. The device of claim 14 , wherein the movable mounting structure includes a rearview mirror.
16. The device of claim 14 , wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, a configuration of the plurality of microphones, a head orientation of the second occupant, or a combination thereof.
17. The device of claim 16 , wherein the zoom direction, the zoom depth, or both, are based on the first location of the first occupant.
18. The device of claim 16 , further comprising an input device coupled to the one or more processors, wherein the one or more processors are configured to receive, via the input device, a user input indicating the zoom direction, the zoom depth, the first location of the first occupant, or a combination thereof.
19. The device of claim 1 , wherein the magnitude of the enhanced audio signal is combined with the first phase based on a first magnitude of the first audio signal and a second magnitude of the second audio signal.
20. The device of claim 1 , wherein the magnitude of the enhanced audio signal is combined with the second phase based on a first magnitude of the first audio signal and a second magnitude of the second audio signal.
21. The device of claim 1 , wherein the audio zoomed signal includes a binaural audio zoomed signal.
22. The device of claim 1 , wherein the one or more processors are configured to apply the spatial filtering based on a zoom direction, a zoom depth, or both.
23. The device of claim 22 , wherein the one or more processors are configured to receive a user input indicating the zoom direction, the zoom depth, or both.
24. The device of claim 22 , further comprising a depth sensor coupled to the one or more processors, wherein the one or more processors are configured to:
receive a user input indicating a zoom target;
receive sensor data from the depth sensor; and
determine, based on the sensor data, the zoom direction, the zoom depth, or both, of the zoom target.
25. The device of claim 24 , wherein the depth sensor includes an image sensor, wherein the sensor data includes image data, and wherein the one or more processors are configured to perform image recognition on the image data to determine the zoom direction, the zoom depth, or both, of the zoom target.
26. The device of claim 24 , wherein the depth sensor includes an ultrasound sensor, a stereo camera, a time-of-flight sensor, an antenna, or a combination thereof.
27. The device of claim 24 , wherein the depth sensor includes a position sensor, wherein the sensor data includes position data indicating a position of the zoom target, and wherein the one or more processors are configured to determine the zoom direction, the zoom depth, or both, of the zoom target based on the position of the zoom target.
28. The device of claim 22 , wherein the one or more processors are configured to determine the zoom depth including:
applying the spatial filtering to the selected audio signals based on the zoom direction and a first zoom depth to generate the first enhanced audio signal;
applying the spatial filtering to the selected audio signals based on the zoom direction and a second zoom depth to generate the second enhanced audio signal; and
based on determining that a first energy of the first enhanced audio signal is less than or equal to a second energy of the second enhanced audio signal, selecting the first enhanced audio signal as the enhanced audio signal and the first zoom depth as the zoom depth.
29. The device of claim 28 , wherein applying the spatial filtering based on the zoom direction and the first zoom depth includes applying the spatial filtering based on a first set of directions of arrival, and wherein applying the spatial filtering based on the zoom direction and the second zoom depth includes applying the spatial filtering based on a second set of directions of arrival.
30. The device of claim 22 , wherein the one or more processors are configured to select the selected audio signals based on the zoom direction, the zoom depth, or both.
31. The device of claim 1 , wherein the one or more processors are configured to the enhanced audio signal based on determining that a first energy of the enhanced audio signal is less than or equal to a second energy of the other of the first enhanced audio signal or the second enhanced audio signal.
32. The device of claim 1 , wherein the one or more processors are configured to apply the spatial filtering to one of the first subset or the second subset with head shade effect correction.
33. The device of claim 1 , wherein the first phase is indicated by first phase values, and wherein each of the first phase values represents a phase of a particular frequency subband of the first audio signal.
34. The device of claim 1 , wherein the one or more processors are configured to generate each of the first output signal and the second output signal based at least in part on a first magnitude of the first audio signal, wherein the first magnitude is indicated by first magnitude values, and wherein each of the first magnitude values represents a magnitude of a particular frequency subband of the first audio signal.
35. The device of claim 1 , wherein the magnitude of the enhanced audio signal is indicated by third magnitude values, and wherein each of the third magnitude values represents a magnitude of a particular frequency subband of the enhanced audio signal.