Deep neural net based filter prediction for audio event classification and extraction
Disclosed is a feature extraction and classification methodology wherein audio data is gathered in a target environment under varying conditions. From this collected data, corresponding features are extracted, labeled with appropriate filters (e.g., audio event descriptions), and used for training deep neural networks (DNNs) to extract underlying target audio events from unlabeled training data. Once trained, these DNNs are used to predict underlying events in noisy audio to extract therefrom features that enable the separation of the underlying audio events from the noisy components thereof.
1. A method for feature extraction and classification of audio signals using deep neural network based filter prediction, the method comprising:
collecting audio event data under varying conditions;
extracting features from the collected audio event data;
labeling the extracted features with filter information;
training a deep neural network to perform filter gain prediction using the extracted features; and
performing filter gain prediction on a target audio event using the trained deep neural network comprising:
applying a predicted filter to an input spectrogram to produce a first output;
applying the predicted filter to the first output to produce a second output; and
determining segmentation of the target audio event based at least in part upon the second output.
2. The method of claim 1 , wherein performing filter gain prediction further comprises processing the target audio event using a noise reference.
3. The method of claim 1 , wherein performing filter gain prediction further comprises:
applying the predicted filter to the second output to produce a third output; and
comparing the second output to the third output to determine if segmentation has improved and:
if segmentation has improved, then iteratively processing the third output for a subsequent compare, and
if segmentation has not improved, then outputting the third output.
4. The method of claim 1 , wherein the training the deep neural network comprises:
acquiring audio event data for feature extraction;
computing labels for a plurality of output layers corresponding to the deep neural network; and
training the deep neural network using the acquired audio event data and the computed labels.
5. The method of claim 4 , wherein the training the deep neural network is iterative until the deep neural network predicts a filter gain meeting a predefined threshold.
6. The method of claim 1 , wherein collecting audio event data is performed using a distributed array of microphones.
7. The method of claim 1 , wherein the extracted features comprise normalized spectral band energies for training the deep neural network.
8. The method of claim 1 , wherein the deep neural network is trained to perform filter gain prediction for target audio extraction by creating a targeted audio event stream using a predicted filter and creating a noise stream using a converse of the predicted filter.
9. A system for deep neural network based filter prediction for audio event classification and extraction, the system comprising:
a memory; and
a processor configured to:
collect audio event data under varying conditions;
extract features from the collected audio event data;
label the extracted features with filter information;
train a deep neural network to perform filter gain prediction using the extracted features; and
perform filter gain prediction on a target audio event using the trained deep neural network comprising:
applying a predicted filter to an input spectrogram to produce a first output;
applying the predicted filter to the first output to produce a second output; and
determining segmentation of the target audio event based at least in part upon the second output.
10. The system of claim 9 , wherein the processor is further configured to process the target audio event using a noise reference.
11. The system of claim 9 , wherein the processor configured to perform the filter gain prediction is further configured to:
apply the predicted filter to the second output to produce a third output; and
compare the second output to the third output to determine if segmentation has improved and:
if segmentation has improved, then iteratively process the third output for a subsequent compare, and
if segmentation has not improved, then output the third output.
12. The system of claim 9 , wherein the processor configured to train the deep neural network comprises the processor configured to:
acquire audio event data for feature extraction;
compute labels for a plurality of output layers corresponding to the deep neural network; and
train the deep neural network using the acquired audio event data and the computed labels.
13. The system of claim 12 , wherein the processor configured to train comprises the processor configured to perform iteratively until the deep neural network predicts a filter gain meeting a predefined threshold.
14. The system of claim 9 , wherein the processor configured to collect audio event data comprises the processor configured to use a distributed array of microphones operatively coupled to the processor.
15. The system of claim 9 , wherein the extracted features comprise normalized spectral band energies for training the deep neural network.
16. An apparatus for feature extraction and classification of audio signals using deep neural network based filter prediction, the apparatus comprising:
means for collecting audio event data under varying conditions;
means for extracting features from the collected audio event data;
means for labeling the extracted features with filter information;
means for training a deep neural network to perform filter gain prediction using the extracted features; and
means for performing filter gain prediction on a target audio event using the trained deep neural network comprising:
means for applying a predicted filter to an input spectrogram to produce a first output;
means for applying the predicted filter to the first output to produce a second output; and
means for determining segmentation of the target audio event based at least in part upon the second output.
17. The apparatus of claim 16 , further comprising means for processing the target audio event using a noise reference.
18. The apparatus of claim 16 , wherein the means for performing filter gain prediction further comprises:
means for applying the predicted filter to the second output to yield a third output; and
means for comparing the second output to the third output to determine if segmentation has improved and:
means for iteratively processing the third output for a subsequent compare if segmentation has improved, and
means for outputting the third output if segmentation has not improved.
19. The apparatus of claim 16 , wherein the means for training the deep neural network comprises:
means for acquiring audio event data for feature extraction;
means for computing labels for a plurality of output layers corresponding to the deep neural network; and
means for training the deep neural network using the acquired audio event data and the computed labels.
20. The apparatus of claim 19 , wherein the means for training is configured to perform iteratively until the deep neural network predicts a filter gain meeting a predefined threshold.
21. The apparatus of claim 16 , wherein the means for collecting audio event data comprises a distributed array of microphones.
22. The apparatus of claim 16 , wherein the extracted features comprise normalized spectral band energies for training the deep neural network.
23. A non-transitory computer readable medium comprising computer-executable instructions for feature extraction and classification of audio signals using deep neural network based filter prediction, the computer-executable instructions comprising instructions for:
collecting audio event data under varying conditions;
extracting features from the collected audio event data;
labeling the extracted features with filter information;
training a deep neural network to perform filter gain prediction using the extracted features; and
performing filter gain prediction on a target audio event using the trained deep neural network comprising:
applying a predicted filter to an input spectrogram to produce a first output;
applying the predicted filter to the first output to produce a second output; and
determining segmentation of the target audio event based at least in part upon the second output.
24. The computer readable medium of claim 23 , wherein the instructions for performing filter gain prediction further comprise instructions for processing the target audio event using a noise reference.
25. The computer readable medium of claim 24 , wherein the instructions for filter gain prediction further comprises instructions for:
applying the predicted filter to the second output to yield a third output; and
comparing the second output to the third output to determine if segmentation has improved and:
if segmentation has improved, iteratively processing the third output for a subsequent compare, and
if segmentation has not improved, then outputting the third output.
26. The computer readable medium of claim 23 , wherein the instructions for training the deep neural network further comprises instructions for:
acquiring audio event data for feature extraction;
computing labels for a plurality of output layers corresponding to the deep neural network; and
training the deep neural network using the acquired audio event data and the computed labels.
27. The computer readable medium of claim 26 , wherein training is iterative until the deep neural network predicts a filter gain meeting a predefined threshold.
28. The computer readable medium of claim 23 , further comprising instructions for collecting audio event data using a distributed array of microphones.
29. The computer readable medium of claim 23 , further comprising instructions for the extracted.