Real-time single-channel speech enhancement in noisy and time-varying environments

1. A method for processing an audio signal in a reverberant environment comprising:

receiving an input signal comprising a time-domain, single-channel audio signal comprising an unknown source signal and a reverberation component;

transforming the input signal to a frequency domain input signal comprising a plurality of k-spaced under-sampled sub-band signals;

reducing a reverberation effect, including late reverberation, in the plurality of k-spaced under-sampled sub-band signals, wherein reducing the reverberation effect comprises:

generating a reverberation prediction filter in real time by blindly processing, with respect to the reverberant environment, the unknown source signal and the reverberation component in the plurality of k-spaced under-sampled sub-band signals, including estimating a short time magnitude spectral density (STMSD) for the late reverberation for a current frame; and

applying the reverberation prediction filter to the plurality of k-spaced under-sampled sub-band signals to suppress the reverberation component;

reducing background noise from the plurality of k-spaced under-sampled sub-band signals; and

transforming the plurality of k-spaced under-sampled sub-band signals to the time-domain, thereby producing an enhanced output signal.

2. The method of claim 1 , wherein reducing the reverberation effect further comprises using spectral subtraction comprising buffering L k frames of the plurality of k-spaced under-sampled sub-band signals, averaging the STMSD over the L k frames, and nonlinearly filtering the plurality of k-spaced under-sampled sub-band signals.

3. The method of claim 2 , further comprising buffering, in a real-value buffer, for each frequency bin a magnitude of spectral density of the input signal for a previous L k frames, and wherein the estimating the STMSD comprises accessing the real-value buffer to estimate the STMSD of the late reverberation.

4. The method of claim 2 , further comprising:

estimating spectral gain for reverberation reduction using Signal To Reverberation Ratio (SRR) and spectral gain floor to reduce distortion in the enhanced output signal; and

applying the estimated spectral gain to reduce the reverberation effect.

5. The method of claim 1 , wherein reducing background noise from the plurality of k-spaced under-sampled sub-band signals further comprises using spectral subtraction which comprises estimating short time power spectral density (STPSD) of noise, estimating spectral gain and nonlinearly filtering the plurality of k-spaced under-sampled sub-band signals.

6. The method of claim 5 , further comprising:

estimating spectral gain for noise reduction using SRR and spectral gain floor to reduce distortion in the enhanced output signal; and

applying noise-reduction spectral gain to reduce background noise, wherein estimating the STPSD further comprises estimating in real time the STPSD of noise.

7. A system for processing an audio signal in a reverberant environment comprising:

a microphone configured to receive an input signal comprising a time-domain, single-channel audio signal comprising an unknown source signal and a reverberation component;

a processor; and

a memory storing instructions that, when executed by the processor, cause the system to:

transform the input signal to a frequency domain input signal comprising a plurality of k-spaced under-sampled sub-band signals;

reduce a reverberation effect, including late reverberation, in the plurality of k-spaced under-sampled sub-band signals, wherein reducing the reverberation effect comprises:

generating a reverberation prediction filter in real time by blindly processing, with respect to the reverberant environment, the unknown source signal and the reverberation component in the plurality of k-spaced under-sampled sub-band signals, including estimating a short time magnitude spectral density (STMSD) of the late reverberation for a current frame; and

applying the reverberation prediction filter to the plurality of k-spaced under-sampled sub-band signals to suppress the reverberation component;

reduce background noise from the plurality of k-spaced under-sampled sub-band signals; and

transform the plurality of k-spaced under-sampled sub-band signals to the time-domain, thereby producing an enhanced output signal.

8. The system of claim 7 , wherein reducing the reverberation effect further comprises using spectral subtraction comprising buffering L k frames of the plurality of k-spaced under-sampled sub-band signals, averaging the STMSD over the L k frames, and nonlinearly filtering the plurality of k-spaced under-sampled sub-band signals.

9. The system of claim 8 , further comprising a real-value buffer storing for each frequency bin a magnitude of spectral density of the input signal for a previous L k frames, wherein estimating the STMSD comprises accessing the real-value buffer to estimate the STMSD of the late reverberation.

10. The system of claim 8 , wherein execution of the instruction further causes the system to:

estimate spectral gain for reverberation reduction using Signal To Reverberation Ratio (SRR) and spectral gain floor to reduce distortion in the enhanced output signal; and apply the estimated spectral gain to reduce the reverberation effect.

11. The system of claim 7 , wherein reducing background noise from the plurality of k-spaced under-sampled sub-band signals further comprises using spectral subtraction which comprises estimating short time power spectral density (STPSD) of noise, estimating spectral gain and nonlinearly filtering the plurality of k-spaced under-sampled sub-band signals.

12. The system of claim 11 , wherein execution of the instructions further causes the system to:

estimate spectral gain for noise reduction using SRR and spectral gain floor to reduce distortion in the enhanced output signal; and

apply noise-reduction spectral gain to reduce background noise, wherein the STPSD is estimated by estimating in real time the STPSD of noise.

13. A method for processing an audio signal in a reverberant environment comprising:

receiving a single-channel audio input signal comprising an unknown source signal and a reverberation component representing reflections of a source in the reverberant environment;

generating a reverberation prediction filter by blindly processing, with respect to the reverberant environment, the unknown source signal and the reverberation component of the single-channel input signal in a frequency domain; and

applying the reverberation prediction filter to the single-channel input signal to suppress the reverberation component and generate a single-channel audio output signal comprising an enhanced source component.

14. The method of claim 13 , wherein an impulse response of the reverberant environment varies over time based, at least in part, on movement of the source; and

wherein generating the reverberation prediction filter further comprises adapting the reverberation prediction filter in real-time to the time-varying impulse response of the reverberant environment.

15. The method of claim 14 , wherein the single-channel input signal further comprises a noise component and wherein the method further comprises reducing the noise component through spectral subtraction, including estimating and applying a spectral noise-reduction gain using non-linear filtering.

16. The method of claim 13 , further comprising:

decomposing the single-channel audio input signal into a plurality of sub-band signals; and

synthesizing the plurality of sub-band signals to produce the single-channel audio output signal, wherein generating the reverberation prediction filter and applying the reverberation prediction filter are performed on the plurality of sub-band signals.

17. The method of claim 16 , wherein each of the plurality of sub-band signals comprises a k-spaced under-sampled sub-band signal.

18. The method of claim 13 , wherein the reverberation component further includes an early reverberation component representing the reflections of the source received within a first period, and a late reverberation component representing the reflections of the source received after the first period; and

wherein generating the reverberation prediction filter further comprises estimating the early reverberation component and the late reverberation component, wherein estimating the late reverberation component comprises estimating a short time magnitude spectral density (STMSD) for a current frame, and generating a nonlinear filter based on the STMSD estimation to reduce the late reverberation component in the current frame.

19. The method of claim 18 , wherein estimating the STMSD of the late reverberation further comprises estimating the reverberation prediction filter using a Rayleigh distribution having tunable parameters.

20. The system of claim 7 , wherein an impulse response of the reverberant environment varies over time based, at least in part, on movement of the source and/or the system; and

wherein reducing reverberation further comprises adapting the reverberation prediction filter in real-time to the time-varying impulse response of the reverberant environment.