Audio-based detection and tracking of emergency vehicles
Techniques are provided for audio-based detection and tracking of an acoustic source. A methodology implementing the techniques according to an embodiment includes generating acoustic signal spectra from signals provided by a microphone array, and performing beamforming on the acoustic signal spectra to generate beam signal spectra, using time-frequency masks to reduce noise. The method also includes detecting, by a deep neural network (DNN) classifier, an acoustic event, associated with the acoustic source, in the beam signal spectra. The DNN is trained on acoustic features associated with the acoustic event. The method further includes performing pattern extraction, in response to the detection, to identify time-frequency bins of the acoustic signal spectra that are associated with the acoustic event, and estimating a motion direction of the source relative to the array of microphones based on Doppler frequency shift of the acoustic event calculated from the time-frequency bins of the extracted pattern.
1. A processor-implemented method for audio-based detection and tracking of an acoustic source, the method comprising:
performing, by a processor-based system, beamforming on a plurality of acoustic signal spectra to generate at least a first beam signal spectrum in a first direction and a second beam signal spectrum in a second direction different from the first direction, the acoustic signal spectra generated from acoustic signals received from an array of microphones;
detecting, by the processor-based system, using a deep neural network (DNN) classifier, an acoustic event associated with the acoustic source in at least one of the first beam signal spectrum or the second beam signal spectrum;
estimating, by the processor-based system, using the deep neural network (DNN) classifier, a direction of the acoustic source of at least one of the first direction or the second direction;
performing, by the processor-based system:
first pattern extraction instructions in response to the detection of the acoustic event, the first pattern extraction instructions associated with a first power state of the processor-based system; and
second pattern extraction instructions prior to the detection of the acoustic event, the second pattern extraction instructions associated with a second power state of the processor-based system, the second power state consuming less power than the first power state, the pattern comprising identified time and frequency bins of the plurality of acoustic signal spectra, the bins associated with the acoustic event; and
estimating, by the processor-based system, a direction of motion of the acoustic source relative to the array of microphones, the estimation based on a Doppler frequency shift of the acoustic event, the Doppler frequency shift calculated from the time and frequency bins of the extracted pattern.
2. The method of claim 1 , further comprising:
applying a Generalized Cross Correlation Phase Transform to the plurality of acoustic signal spectra to generate an angular spectrum; and
estimating the direction of the acoustic source relative to the array of microphones based on detection of a peak in the angular spectrum.
3. The method of claim 1 , wherein the pattern extraction comprises comparing one or more of the plurality of acoustic signal spectra to a predetermined spectrum associated with an expected pattern, and identifying time and frequency bins that match, based on the comparison, to within a threshold value.
4. The method of claim 1 , wherein the pattern extraction comprises applying a neural network to one or more of the plurality of acoustic signal spectra, the neural network trained to generate scores for time and frequency bins of the acoustic signal spectra that indicate a probability of matching to an acoustic event of interest.
5. The method of claim 1 , wherein the acoustic source is an emergency vehicle and the acoustic event is a siren.
6. The method of claim 1 , further comprising applying a high-pass filter to the acoustic signal spectra to reduce wind noise.
7. The method of claim 1 , further including calculating the Doppler frequency shift based on the time and frequency bins of the extracted pattern, a known frequency of the acoustic event, and a velocity of an autonomous vehicle.
8. At least one non-transitory computer readable storage medium comprising instructions encoded thereon that, when executed, cause one or more processors to at least:
perform beamforming on a plurality of acoustic signal spectra to generate at least a first beam signal spectrum in a first direction and a second beam signal spectrum in a second direction different from the first direction, the acoustic signal spectra generated from acoustic signals received from an array of microphones;
detect, by a deep neural network (DNN) classifier, an acoustic event associated with an acoustic source, in at least one of the first beam signal spectrum or the second beam signal spectrum;
estimate, by the deep neural network (DNN) classifier, a direction of the acoustic source of at least one of the first direction or the second direction;
perform:
first pattern extraction in response to the detection of the acoustic event, the first pattern extraction associated with a first power state of the one or more processors; and
second pattern extraction prior to the detection of the acoustic event, the second pattern extraction associated with a second power state of the one or more processors, the second power state consuming less power than the first power state, the pattern comprising identified time and frequency bins of the plurality of acoustic signal spectra, the bins associated with the acoustic event; and
estimate a direction of motion of the acoustic source relative to the array of microphones, the estimation based on a Doppler frequency shift of the acoustic event, the Doppler frequency shift calculated from the time and frequency bins of the extracted pattern.
9. The computer readable storage medium of claim 8 , wherein the instructions cause the one or more processors to:
apply a Generalized Cross Correlation Phase Transform to the plurality of acoustic signal spectra to generate an angular spectrum; and
estimate a direction of the acoustic source relative to the array of microphones based on detection of a peak in the angular spectrum.
10. The computer readable storage medium of claim 8 , wherein the instructions cause the one or more processors to compare one or more of the plurality of acoustic signal spectra to a predetermined spectrum associated with an expected pattern, and identify time and frequency bins that match, based on the comparison, to within a threshold value when performing the pattern extraction.
11. The computer readable storage medium of claim 8 , wherein the instructions cause the one or more processors to apply a neural network to one or more of the plurality of acoustic signal spectra, the neural network trained to generate scores for time and frequency bins of the acoustic signal spectra that indicate a probability of matching to an acoustic event of interest, when performing the pattern extraction.
12. The computer readable storage medium of claim 8 , wherein the acoustic source is an emergency vehicle and the acoustic event is a siren.
13. The computer readable storage medium of claim 8 , wherein the instructions cause the one or more processors to apply a high-pass filter to the acoustic signal spectra to reduce wind noise.
14. A system for audio-based detection and tracking of an acoustic source, the system comprising:
a beamforming circuit to perform beamforming on a plurality of acoustic signal spectra to generate at least a first beam signal spectrum in a first direction and a second beam signal spectrum in a second direction different from the first direction, the acoustic signal spectra generated from acoustic signals received from an array of microphones;
a deep neural network (DNN) classifier to:
detect an acoustic event associated with the acoustic source, in at least one of the first beam signal spectrum or the second beam signal spectrum; and
estimate a direction of the acoustic source of at least one of the first direction or the second direction;
a pattern extraction circuit to perform:
first pattern extraction instructions in response to the detection of the acoustic event, the first pattern extraction instructions associated with a first power state of the pattern extraction circuit; and
second pattern extraction instructions prior to the detection of the acoustic event, the second pattern extraction instructions associated with a second power state of the pattern extraction circuit, the second power state consuming less power than the first power state, the pattern comprising identified time and frequency bins of the plurality of acoustic signal spectra, the bins associated with the acoustic event; and
a movement direction estimation circuit to estimate a direction of motion of the acoustic source relative to the array of microphones, the estimation based on a Doppler frequency shift of the acoustic event, the Doppler frequency shift calculated from the time and frequency bins of the extracted pattern.
15. The system of claim 14 , further comprising a direction of arrival estimation circuit to:
apply a Generalized Cross Correlation Phase Transform to the plurality of acoustic signal spectra to generate an angular spectrum; and
estimate a direction of the acoustic source relative to the array of microphones based on detection of a peak in the angular spectrum.
16. The system of claim 14 , wherein the pattern extraction circuit is further to compare one or more of the plurality of acoustic signal spectra to a predetermined spectrum associated with an expected pattern, and identify time and frequency bins that match, based on the comparison, to within a threshold value.
17. The system of claim 14 , wherein the pattern extraction circuit further comprises a neural network for application to one or more of the plurality of acoustic signal spectra, the neural network trained to generate scores for time and frequency bins of the acoustic signal spectra that indicate a probability of matching to an acoustic event of interest.
18. The system of claim 14 , wherein the acoustic source is an emergency vehicle and the acoustic event is a siren.
19. The system of claim 14 , further comprising a signal conditioning circuit to apply a high-pass filter to the acoustic signal spectra to reduce wind noise.
20. The system of claim 14 , wherein a magnitude of the direction of motion causes the acoustic source to undertake a safety related action, the safety related action including at least one of pulling over an autonomous vehicle on to a shoulder or stop the autonomous vehicle.