Joint beamforming and echo cancellation for reduction of noise and non-linear echo
Techniques are provided for reduction of noise and nonlinear-echo. A methodology implementing the techniques according to an embodiment includes estimating transfer functions (TFs) of echo paths of audio signals received through a microphone array. The audio signals include speech signal, additive noise, and echo, the TF estimation based on the reference signal. The method also includes cancellation of linear components of the echo, based on the echo path TFs. The method further includes estimating an inverse square root of a covariance matrix of the additive noise, whitening the echo cancelled signals, and estimating a speech path RTF associated with the speech signal, based on the whitened echo cancelled signals. The method further includes performing beamforming on the whitened signals (such as weighted MVDR beamforming), based on the echo path TFs, the speech path RTF, and the estimated inverse square root additive noise covariance matrix.
1. A processor-implemented method for reducing noise and echo in an audio signal, the method comprising:
estimating, by a processor-based system, a transfer function (TF) of an echo path associated with a received audio signal, the audio signal including a combination of a speech signal, additive noise, and an echo signal, the estimation based on a reference signal;
performing, by the processor-based system, cancellation of one or more linear components of the echo signal, based on the echo path TF, to provide an echo cancelled signal;
estimating, by the processor-based system, a square root of an inverse of a covariance matrix of the additive noise;
whitening, by the processor-based system, the echo cancelled signal;
estimating, by the processor-based system, a speech path relative transfer function (RTF) associated with the speech signal, based on the whitened echo cancelled signal; and
performing, by the processor-based system, beamforming on the whitened echo cancelled signal, based on the echo path TF, the speech path RTF, and the estimated square root of the inverse of the covariance matrix of the additive noise.
2. The method of claim 1 , wherein the estimation of the echo path TF employs a Recursive Least Squares (RLS)-Inverse QR Decomposition (IQRD).
3. The method of claim 1 , wherein the estimation of the square root of the inverse of the covariance matrix of the additive noise employs an RLS-IQRD.
4. The method of claim 1 , wherein the beamforming is weighted Minimum Variance Distortionless Response (MVDR) beamforming, the method further comprising generating the echo signal to include non-linear distortion components, the MVDR beamforming further to reduce the non-linear distortion components of the echo signal.
5. The method of claim 1 , wherein the estimating of the speech path RTF is performed during time periods associated with the presence of the speech signal and the absence of the echo signal.
6. The method of claim 1 , wherein the processor-based system is a smartphone and the echo signal is generated by a loudspeaker of the smartphone during a voice call in speakerphone mode.
7. The method of claim 1 , wherein the processor-based system is a smart-speaker system and the echo signal is generated by playing selected audio content.
8. A system for reducing noise and echo in an audio signal, the system comprising:
an echo path transfer function (TF) estimation circuit to estimate the TF of an echo path associated with a received audio signal, the audio signal including a combination of a speech signal, additive noise, and an echo signal, the estimation based on a reference signal;
an echo canceller application circuit to cancel one or more linear components of the echo signal, based on the echo path TF, to provide an echo cancelled signal;
a matrix square root estimation circuit to estimate a square root of an inverse of a covariance matrix of the additive noise;
a whitening circuit to whiten the echo cancelled signal;
a speech path estimation circuit to estimate a speech path relative transfer function (RTF) associated with the speech signal, based on the whitened echo cancelled signal; and
a spatial filtering circuit to perform beamforming on the whitened echo cancelled signal, based on the echo path TF, the speech path RTF, and the estimated square root of the inverse of the covariance matrix of the additive noise.
9. The system of claim 8 , wherein the echo path TF estimation circuit is further to estimate the echo path TF based on a Recursive Least Squares (RLS)-Inverse QR Decomposition (IQRD).
10. The system of claim 8 , wherein the matrix square root estimation circuit is further to estimate the square root of the inverse of the covariance matrix of the additive noise based on an RLS-IQRD.
11. The system of claim 8 , wherein the beamforming is weighted Minimum Variance Distortionless Response (MVDR) beamforming, the system further comprising a loudspeaker to generate the echo signal to include non-linear distortion components, the spatial filtering circuit further to reduce the non-linear distortion components of the echo signal.
12. The system of claim 8 , wherein the estimating of the speech path RTF is performed during time periods associated with the presence of the speech signal and the absence of the echo signal.
13. The system of claim 8 , wherein the system is a smartphone and the echo signal is generated by a loudspeaker of the smartphone during a voice call in speakerphone mode.
14. The system of claim 8 , wherein the system is a smart-speaker system and the echo signal is generated by playing selected audio content.
15. At least one non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, cause a process to be carried out for reducing noise and echo in an audio signal, the process comprising:
estimating a transfer function (TF) of an echo path associated with a received audio signal, the audio signal including a combination of a speech signal, additive noise, and an echo signal, the estimation based on a reference signal;
performing cancellation of one or more linear components of the echo signal, based on the echo path TF, to provide an echo cancelled signal;
estimating a square root of an inverse of a covariance matrix of the additive noise;
whitening the echo cancelled signal;
estimating a speech path relative transfer function (RTF) associated with the speech signal, based on the whitened echo cancelled signal; and
performing beamforming on the whitened echo cancelled signal, based on the echo path TF, the speech path RTF, and the estimated square root of the inverse of the covariance matrix of the additive noise.
16. The computer readable storage medium of claim 15 , wherein the estimation of the echo path TF comprises a Recursive Least Squares (RLS)-Inverse QR Decomposition (IQRD) operation.
17. The computer readable storage medium of claim 15 , wherein the estimation of the square root of the inverse of the covariance matrix of the additive noise comprises an RLS-IQRD operation.
18. The computer readable storage medium of claim 15 , wherein the beamforming is weighted Minimum Variance Distortionless Response (MVDR) beamforming, the computer readable storage medium further comprising the operation of generating the echo signal to include non-linear distortion components, the MVDR beamforming further to reduce the non-linear distortion components of the echo signal.
19. The computer readable storage medium of claim 15 , wherein the estimating of the speech path RTF is performed during time periods associated with the presence of the speech signal and the absence of the echo signal.
20. The computer readable storage medium of claim 15 , wherein the processor-based system is a smartphone and the echo signal is generated by a loudspeaker of the smartphone during a voice call in speakerphone mode.
21. The computer readable storage medium of claim 15 , wherein the processor-based system is a smart-speaker system and the echo signal is generated by playing selected audio content.