System and method for multi-channel multi-feature speech/noise classification for noise suppression
View Patent ↗An architecture and framework for speech/noise classification of an audio signal using multiple features with multiple input channels (e.g., microphones) are provided. The architecture may be implemented with noise suppression in a multi-channel environment where noise suppression is based on an estimation of the noise spectrum. The noise spectrum is estimated using a model that classifies each time/frame and frequency component of a signal as speech or noise by applying a speech/noise probability function. The speech/noise probability function estimates a speech/noise probability for each frequency and time bin. A speech/noise classification estimate is obtained by fusing (e.g., combining) data across different input channels using a layered network model. Individual feature data acquired at each channel and/or from a beam-formed signal is mapped to a speech probability, which is combined through layers of the model into a final speech/noise classification for use in noise estimation and filtering processes for noise suppression.
1. A method for noise estimation and filtering based on classifying an audio signal received at a noise suppression module via a plurality of input channels as speech or noise, the method comprising:
measuring signal classification features for a frame of the audio signal input from each of the plurality of input channels;
generating a feature-based speech probability for each of the measured signal classification features of each of the plurality of input channels;
generating a combined speech probability for the measured signal classification features over the plurality of input channels using a probabilistic layered network model, wherein an additive model is used for a top layer of the probabilistic layered network model;
classifying the audio signal as speech or noise based on the combined speech probability; and
updating an initial noise estimate for each of the plurality of input channels using the combined speech probability.
2. The method of claim 1 , wherein the measured signal classification features from the plurality of input channels are input data to the probabilistic layered network model.
3. The method of claim 1 , wherein the combined speech probability over the plurality of input channels is an output of the probabilistic layered network model.
4. The method of claim 1 , wherein the probabilistic layered network model includes a set of intermediate states each denoting a class state of speech or noise for one or more layers of the probabilistic layered network model.
5. The method of claim 4 , wherein the probabilistic layered network model further includes a set of state-conditioned transition probabilities.
6. The method of claim 5 , wherein the speech probability for the intermediate state of the layer of the probabilistic layered network model is determined using one or both of an additive model and a multiplicative model.
7. The method of claim 4 , wherein the feature-based speech probability for each of the measured signal classification features denotes a probability of a class state of speech or noise for a layer of the one or more layers of probabilistic layered network model.
8. The method of claim 4 , further comprising determining a speech probability for an intermediate state of a layer of the probabilistic layered network model using data from a lower layer of the probabilistic layered network model.
9. The method of claim 4 , further comprising generating, for each of the plurality of input channels, a speech probability for the input channel using the feature-based speech probabilities of the input channel.
10. The method of claim 9 , wherein the feature-based speech probability is a function of the measured signal classification feature, and wherein the speech probability for each of the plurality of input channels is a function of the feature-based speech probabilities for the input channel.
11. The method of claim 1 , wherein classifying the audio signal as speech or noise based on the combined speech probability includes applying a threshold to the combined speech probability.
12. The method of claim 1 , further comprising determining an initial noise estimate for each of the plurality of input channels.
13. The method of claim 1 , further comprising:
combining the frames of the audio signal input from the plurality of input channels;
measuring at least one signal classification feature of the combined frames of the audio signal;
calculating a feature-based speech probability for the combined frames using the measured at least one signal classification feature; and
combining the feature-based speech probability for the combined frames with the speech probabilities generated for each of the plurality of input channels.
14. The method of claim 13 , wherein the combined frames of the audio signal is a time-aligned superposition of the frames of the audio signal received at each of the plurality of input channels.
15. The method of claim 13 , wherein the combined frames of the audio signal is a signal generated using beam-forming on signals from the plurality of input channels.
16. The method of claim 13 , wherein the combined frames of the audio signal is used as an additional input channel to the plurality of input channels.
17. The method of claim 1 , wherein the initial noise estimate is updated with a recursive time average using a combined speech probability function.
18. The method of claim 17 , wherein updating the initial noise estimate with the recursive time average includes using an input magnitude spectrum quantity to weight the speech probability, the input magnitude spectrum quantity being a magnitude spectrum of one of the plurality of input channels, a magnitude spectrum of the combined frames, or a combination of the magnitude spectrums of one of the plurality of input channels and the combined frames.
19. The method of claim 1 , wherein the feature-based speech probability is generated for each of the signal classification features by mapping each of the signal classification features to a probability value using a map function.
20. The method of claim 19 , wherein the map function is a model with a set of width and threshold parameters.
21. The method of claim 19 , wherein the feature-based speech probability is updated with a time-recursive average.
22. The method of claim 1 , wherein the signal classification features include at least: average likelihood ratio over time, spectral flatness measure, and spectral template difference measure.
23. The method of claim 1 , wherein for a single input channel an additive model is used for a middle layer of the probabilistic layered network model to generate a speech probability for the single input channel.
24. The method of claim 1 , wherein for a single input channel a multiplicative model is used for a middle layer of the probabilistic layered network model to generate a speech probability for the single input channel.
25. The method of claim 1 , wherein a state-conditioned transition probability for an intermediate state at any intermediate layer of the probabilistic layered network model is fixed off-line or determined adaptively on-line.
26. The method of claim 1 , wherein a beam-formed signal is another input to the probabilistic layered network model, and wherein the additive model is used for the top layer of the probabilistic layered network model to generate a speech probability for the plurality of input channels and the beam-formed signal.
27. The method of claim 26 , wherein for the beam-formed signal, a speech probability conditioned on signal classification features of the beam-formed signal is obtained by mapping the signal classification features to a probability value using a map function.
28. The method of claim 27 , wherein a time-recursive average is used to update the speech probability of the beam-formed signal.