Method and apparatus for estimating variability of background noise for noise suppression
An electronic device measures noise variability of background noise present in a sampled audio signal, and determines whether the measured noise variability is higher than a high threshold value or lower than a low threshold value. If the noise variability is determined to be higher than the high threshold value, the device categorizes the background noise as having a high degree of variability. If the noise variability is determined to be lower than the low threshold value, the device categorizes the background noise as having a low degree of variability. The high and low threshold values are between a high boundary point and a low boundary point. The high boundary point is based on an analysis of files including noises that exhibit a high degree of variability, and the low boundary point is based on an analysis of files including noises that exhibit a low degree of variability.
1. A computer-implemented method comprising:
receiving audio data that includes speech and background noise;
processing the background noise included in the audio data to:
obtain smoothed channel noise energy associated with the background noise; and
determine a noise level of the background noise;
determining whether a magnitude of the noise level of the background noise satisfies a noise level threshold;
when the magnitude of the noise level of the background noise satisfies the noise level threshold, setting a bias energy level to a predetermined value based on the magnitude of the noise level of the background noise;
adding the bias energy level set to the predetermined value to the smoothed channel noise energy to generate modified smoothed channel noise energy;
determining, using the modified smoothed channel noise energy, a degree of variability of the background noise included in the audio data;
determining that the degree of variability of the background noise included in the audio data satisfies a threshold;
based on determining that the degree of variability of the background noise included in the audio data satisfies the threshold, selecting, from among at least a first noise suppressor that requires a first amount of processing power and a second noise suppressor that requires a second, greater amount of processing power, the second noise suppressor that requires the second, greater amount of processing power;
based on selecting the second noise suppressor that requires the second, greater amount of processing power, processing the audio data using the second noise suppressor that requires the second, greater amount of processing power; and
after the second noise suppressor that requires the second, greater amount of processing power has processed the audio data, obtaining, by performing speech recognition, a transcription of one or more of the portions of the audio data that include speech.
2. The method of claim 1 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power in further response to classifying the background noise as high variability background noise.
3. The method of claim 1 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power in further response to determining that the degree of variability is greater than the threshold.
4. The method of claim 1 , wherein processing the audio data using the second noise suppressor that requires the second, greater amount of processing power comprises processing smoothed audio data using the second noise suppressor that requires the second, greater amount of processing power.
5. The method of claim 1 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power is in further response to determining that the degree of variability is greater than the threshold in further response to:
determining that a respective portion of the audio data that includes the background noise is a noise update portion;
based on determining that the respective portion of the audio data that includes the background noise is a noise update portion, determining that the respective portion is not part of a sequence of contiguous noise portions; and
determining that the respective portion of the audio data has a degree of variability of a previous portion of the audio data.
6. The method of claim 1 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power is in further response to determining that the degree of variability is greater than the threshold in further response to:
determining that a respective portion of the audio data that includes the background noise is not a noise update portion; and
based on determining that the respective portion of the audio data that includes the background noise is not a noise update portion, determining that the respective portion of the audio data has a degree of variability of a previous portion of the audio data.
7. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving audio data that includes speech and background noise;
processing the background noise included in the audio data to:
obtain smoothed channel noise energy associated with the background noise; and
determine a noise level of the background noise;
determining whether a magnitude of the noise level of the background noise satisfies a noise level threshold;
when the magnitude of the noise level of the background noise satisfies the noise level threshold, setting a bias energy level to a predetermined value based on the magnitude of the noise level of the background noise;
adding the bias energy level set to the predetermined value to the smoothed channel noise energy to generate modified smoothed channel noise energy;
determining, using the modified smoothed channel noise energy, a degree of variability of the background noise included in the audio data;
determining that the degree of variability of the background noise included in the audio data satisfies a threshold;
based on determining that the degree of variability of the background noise included in the audio data satisfies the threshold, selecting, from among at least a first noise suppressor that requires a first amount of processing power and a second noise suppressor that requires a second, greater amount of processing power, the second noise suppressor that requires the second, greater amount of processing power;
based on selecting the second noise suppressor that requires the second, greater amount of processing power, processing the audio data using the second noise suppressor that requires the second, greater amount of processing power; and
after the second noise suppressor that requires the second, greater amount of processing power has processed the audio data, obtaining, by performing speech recognition, a transcription of one or more of the portions of the audio data that include speech.
8. The system of claim 7 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power in further response to classifying the background noise as high variability background noise.
9. The system of claim 7 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power in further response to determining that the degree of variability is greater than the threshold.
10. The system of claim 7 , wherein processing the audio data using the second noise suppressor that requires the second, greater amount of processing power comprises processing smoothed audio data using the second noise suppressor that requires the second, greater amount of processing power.
11. The system of claim 7 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power is in further response to determining that the degree of variability is greater than the threshold in further response to:
determining that a respective portion of the audio data that includes the background noise is a noise update portion;
based on determining that the respective portion of the audio data that includes the background noise is a noise update portion, determining that the respective portion is not part of a sequence of contiguous noise portions; and
determining that the respective portion of the audio data has a degree of variability of a previous portion of the audio data.
12. The system of claim 7 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power is in further response to determining that the degree of variability is greater than the threshold in further response to:
determining that a respective portion of the audio data that includes the background noise is not a noise update portion; and
based on determining that the respective portion of the audio data that includes the background noise is not a noise update portion, determining that the respective portion of the audio data has a degree of variability of a previous portion of the audio data.
13. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
receiving audio data that includes speech and background noise;
processing the background noise included in the audio data to:
obtain smoothed channel noise energy associated with the background noise; and
determine a noise level of the background noise;
determining whether a magnitude of the noise level of the background noise satisfies a noise level threshold;
when the magnitude of the noise level of the background noise satisfies the noise level threshold, setting a bias energy level to a predetermined value based on the magnitude of the noise level of the background noise;
adding the bias energy level set to the predetermined value to the smoothed channel noise energy to generate modified smoothed channel noise energy;
determining, using the modified smoothed channel noise energy, a degree of variability of the background noise included in the audio data;
determining that the degree of variability of the background noise included in the audio data satisfies a threshold;
based on determining that the degree of variability of the background noise included in the audio data satisfies the threshold, selecting, from among at least a first noise suppressor that requires a first amount of processing power and a second noise suppressor that requires a second, greater amount of processing power, the second noise suppressor that requires the second, greater amount of processing power;
based on selecting the second noise suppressor that requires the second, greater amount of processing power, processing the audio data using the second noise suppressor that requires the second, greater amount of processing power; and
after the second noise suppressor that requires the second, greater amount of processing power has processed the audio data, obtaining, by performing speech recognition, a transcription of one or more of the portions of the audio data that include speech.
14. The medium of claim 13 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power in further response to classifying the background noise as high variability background noise.
15. The medium of claim 13 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power in further response to determining that the degree of variability is greater than the threshold.
16. The medium of claim 13 , wherein processing the audio data using the second noise suppressor that requires the second, greater amount of processing power comprises processing smoothed audio data using the second noise suppressor that requires the second, greater amount of processing power.
17. The medium of claim 13 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power is in further response to determining that the degree of variability is greater than the threshold in further response to:
determining that a respective portion of the audio data that includes the background noise is a noise update portion;
based on determining that the respective portion of the audio data that includes the background noise is a noise update portion, determining that the respective portion is not part of a sequence of contiguous noise portions; and
determining that the respective portion of the audio data has a degree of variability of a previous portion of the audio data.
18. The medium of claim 13 , wherein the audio data is processed using the second noise suppressor that requires the second, greater amount of processing power is in further response to determining that the degree of variability is greater than the threshold in further response to:
determining that a respective portion of the audio data that includes the background noise is not a noise update portion; and
based on determining that the respective portion of the audio data that includes the background noise is not a noise update portion, determining that the respective portion of the audio data has a degree of variability of a previous portion of the audio data.