Audio acoustic echo cancellation for video conferencing
View Patent ↗A new audio echo cancellation (AEC) approach is disclosed. To facilitate echo cancellation, the method adjusts for errors (called drift) in sampling rates for both capturing audio and playing audio. This ensures that the AEC module receives both the signals at precisely the same sampling frequency. Furthermore, the far-end signal and near-end mixed signal are time aligned to ensure that the alignment is suitable for application of AEC techniques. An additional enhancement to reduce errors utilizes a concept of native frequency. A by-product of drift compensation allows for excellent buffer control for capture/playback and buffer overflow/underflow errors from drift errors are eliminated.
1. A method for audio acoustic echo cancellation in an audio conference with multiple participants, the audio conference implemented over a network, the method implemented at a participant's audio conference device connected to the network, the participant's audio conference device having a speaker and a microphone, the method comprising:
receiving a far-end audio signal effectively sampled at a nominal frequency f nom for the audio conference, the far-end audio signal originating from another audio conference device;
playing back the far-end audio signal through the speaker;
capturing a mixed signal through the microphone, the mixed signal containing an echo of the played back far-end audio signal, the mixed signal captured at a capture frequency f M ;
estimating a mismatch between the capture frequency f M and the nominal frequency f nom ;
adjusting the mixed signal to an effective sampling rate of f nom to compensate for the estimated mismatch; and
applying audio acoustic echo cancellation to the far-end audio signal effectively sampled at f nom and the mixed signal effectively sampled at f nom .
2. The method of claim 1 wherein the step of receiving the far-end audio signal effectively sampled at f nom comprises receiving the far-end audio signal effectively sampled at f nom over the network.
3. The method of claim 1 wherein the step of receiving the far-end audio signal effectively sampled at f nom comprises:
receiving the far-end audio signal effectively sampled at f far over the network; and
converting the sampling rate of the far-end audio signal from f far to an effective sampling rate of f nom .
4. The method of claim 1 wherein the step of receiving the far-end audio signal effectively sampled at f nom comprises:
receiving the far-end audio signal effectively sampled at f far over the network; and
adjusting the far-end audio signal to an effective sampling rate of f nom to compensate for a mismatch between f far and f nom .
5. The method of claim 1 further comprising:
adjusting the far-end audio signal sampled at f nom to an effective sampling rate of f S before playback at the playback frequency f S .
6. The method of claim 1 wherein f nom is a native frequency of the participant's audio conference device.
7. The method of claim 1 wherein the step of estimating a mismatch between the capture frequency f M and the nominal frequency f nom comprises:
estimating a drift between the capture frequency f M and the nominal frequency f nom .
8. The method of claim 6 wherein the step of estimating the drift is based on a linear regression of drift as a function of time step.
9. The method of claim 7 wherein the linear regression is restarted if an estimated error in the linear regression exceeds a threshold.
10. The method of claim 7 wherein a next data point for the linear regression is discarded if it causes an increase in the estimated error that exceeds a threshold.
11. The method of claim 6 wherein the step of adjusting the mixed signal to an effective sampling rate of f nom comprises:
based on the estimated drift, adding or removing samples from the mixed signal.
12. The method of claim 6 wherein the step of adjusting the mixed signal to an effective sampling rate of f nom comprises:
resampling the mixed signal.
13. The method of claim 1 further comprising:
time synchronizing the far-end audio signal effectively sampled at f nom and the mixed signal effectively sampled at f nom , wherein the step of applying audio acoustic echo cancellation occurs after time synchronization.
14. The method of claim 12 wherein the step of time synchronizing comprises adjusting a delay between the far-end audio signal and the mixed signal such that the mixed signal is delayed between 0 and a preselected upper threshold relative to the far-end audio signal.
15. The method of claim 1 wherein the step of applying audio acoustic echo cancellation comprises applying the audio acoustic echo cancellation method of claim 19 .
16. The method of claim 1 wherein the participant's audio conference device is a computer with a sound card.
17. The method of claim 1 wherein the network includes the Internet.
18. The method of claim 1 wherein the audio conference is an audio portion of a video conference.
19. The method of claim 1 wherein the audio conference has at least three participants.
20. A method for audio acoustic echo cancellation in an audio conference with multiple participants, the audio conference implemented over a network, the method implemented at a participant's audio conference device connected to the network, the participant's audio conference device having a speaker and a microphone, the method comprising:
receiving via the network a far-end audio signal;
capturing a mixed signal through the microphone, the mixed signal containing an echo of the far-end audio signal played back through the speaker;
applying a first phase of audio acoustic echo cancellation (AEC) to the far-end audio signal and the mixed signal, the first phase AEC producing an estimate of the echo and reducing the echo in the mixed signal; and
applying a second phase of AEC to the echo-reduced mixed signal, the second phase AEC receiving the estimate of the echo from the first phase and using said estimate as an estimate of the residual echo in the echo-reduced mixed signal.
21. The method of claim 19 wherein the first phase AEC includes a multi delay block frequency domain adaptive filter algorithm.
22. The method of claim 19 wherein the first phase AEC includes a normalized least means square algorithm.
23. The method of claim 19 wherein the first phase AEC includes a least means square algorithm.