Joint signal and model based noise matching noise robustness method for automatic speech recognition
View Patent ↗A noise robustness method operates jointly in a signal domain and a model domain. For example, energy is added in the signal domain for frequency bands where an actual noise level of an incoming signal is lower than a noise level used to train models, thus obtaining a compensated signal. Also, energy is added in the model domain for frequency bands where noise level of the incoming signal or the compensated signal is higher than the noise level used to train the models. Moreover, energy is never removed, thereby avoiding problems of higher sensitivity of energy removal to estimation errors.
1. A noise robustness method operating jointly in a signal domain and a model domain, comprising:
adding energy in frequency bands of the signal domain corresponding to frequency bands of an input signal having an actual noise level that is less than a noise level used to train an acoustic model, thereby obtaining a compensated signal, wherein said input signal is indicative of speech input;
adding energy in frequency bands using a model compensation module of the model domain corresponding to frequency bands of at least one of the input signal and the compensated signal having a noise level that is higher than the noise level used to train the acoustic model, thereby obtaining a noise matched acoustic model.
2. The method of claim 1 , further comprising, for each frequency band of an incoming signal, comparing signal noise level at a particular frequency band to the training noise level at that frequency band.
3. The method of claim 2 , further comprising adding an amount of energy to a frequency band that is equivalent to a magnitude difference at that frequency band between the signal noise level and the training noise level.
4. The method of claim 1 , further comprising extracting features from the incoming signal following adding of energy in the signal domain, thereby obtaining extracted features.
5. The method of claim 4 , further comprising performing pattern matching between the noise matched acoustic model and the extracted features, thereby achieving automatic speech recognition.
6. The method of claim 1 , further comprising employing a multi-conditioned acoustic model.
7. The method of claim 1 , wherein adding energy in the signal domain includes only adding energy for the frequency bands where the actual noise level of the incoming signal is less than the noise level used to train the acoustic model.
8. The method of claim 1 further comprising refraining from removing energy in either of the signal domain and the model domain.
9. An automatic speech recognizer implementing a noise robustness method operating jointly in a signal domain and a model domain, comprising:
a signal-based spectral add matching module adding energy to frequency bands of an input signal having an actual noise level that is lower than a noise level used to train an acoustic model, thereby obtaining a compensated signal; and
a model compensation block adding energy to frequency bands of the acoustic model corresponding to frequency bands of at least one of the incoming signal or the compensated signal having a noise level that is higher than the noise level used to train the acoustic model, thereby obtaining noise matched acoustic model
wherein energy is not removed from frequency bands of the input signal or the acoustic models.
10. The system of claim 9 , wherein said signal-based spectral add matching module adds energy to frequency bands to have noise levels of the trained acoustic model match noise levels of the input signal at those frequencies.
11. The system of claim 9 , wherein said model compensation block adds energy to frequency bands to have noise levels of the trained acoustic model match noise levels of the input signal at those frequencies.
12. The system of claim 9 , further comprising a residual noise estimation module estimating noise levels for frequency bands of the input signal that have more noise than the trained acoustic model at those frequency bands, thereby obtaining frequency specific noise estimates.
13. The system of claim 9 , further comprising a pattern matching module performing pattern matching between the noise matched acoustic model and extracted features obtained from the compensated signal, thereby achieving automatic speech recognition.
14. The system of claim 13 , further comprising extracting features from the compensated signal, thereby obtaining the extracted features.
15. The system of claim 9 , wherein the acoustic model includes a multi-conditioned acoustic model.
16. The system of claim 9 , wherein said signal-based spectral add matching module only adds energy for the frequency bands where the actual noise level of the incoming signal is less than the noise level used to train the model.