Machine-learning assisted acoustic echo cancelation
Example methods and systems provide machine-learning assisted acoustic echo cancellation (AEC). The AEC can be used, as an example, to improve the audio quality for online audio and video conferences. A system according to this disclosure includes a pre-trained, machine-learning, AI model designed to detect, in real time, a unitary voice signal, or a signal representing the speech of a single speaker as opposed to that of multiple speakers. A digital signal processing (DSP) algorithm can then detect the echo state, for example, whether distortion results primarily from an echo. Based on these characteristics, the system can, alternatively and automatically apply either a default mode of AEC to the audio signal, or apply a more aggressive mode of AEC.
1 . A method comprising:
accessing an audio signal;
identifying, using a trained, machine-learning (“ML”) model, the audio signal as a unitary voice signal;
applying a first mode of acoustic echo cancelation (AEC) to an audio frame of the unitary voice signal to produce a test frame;
measuring a residual echo in the test frame to produce a residual value;
comparing the residual value to a threshold; and
applying a second mode of AEC to the audio signal based on the comparing.
2 . The method of claim 1 , wherein the threshold comprises a plurality of thresholds, and the second mode of AEC comprises a plurality of modes with different attenuation levels for an echo in the audio signal.
3 . The method of claim 1 , further comprising:
setting a stored echo flag based on the identifying of the audio signal as a unitary voice signal; and
selectively applying the first mode or the second mode of AEC to the audio signal based on the stored echo flag.
4 . The method of claim 1 , further comprising receiving or updating the trained ML model as configured by training datasets including echo data and single-word data.
5 . The method of claim 4 , wherein:
the echo data comprises recording, clipping, distortion, and room simulation data; and
the single-word data comprises far-end interrupt echo data and near-end interrupt voice data.
6 . The method of claim 4 , wherein the trained ML model comprises:
a plurality of convolutional neural networks with node weights configured by the training datasets;
a classifier configured to identify a single-speaker class; and
a plurality of fully connected layers disposed between the plurality of convolutional neural networks and the classifier.
7 . The method of claim 1 , further comprising post processing the unitary voice signal.
8 . A system comprising:
a processor; and
at least one memory device including instructions that are executable by the processor to cause the processor to:
access an audio signal;
identify, using a trained, machine-learning (“ML”) model, the audio signal as a unitary voice signal;
apply a first mode of acoustic echo cancelation (AEC) to an audio frame of the unitary voice signal to produce a test frame;
measure a residual echo in the test frame to produce a residual value;
compare the residual value to a threshold; and
apply a second mode of AEC to the audio signal based on the comparing.
9 . The system of claim 8 , wherein the threshold comprises a plurality of thresholds, and the second mode of AEC comprises a plurality of modes with different attenuation levels for an echo in the audio signal.
10 . The system of claim 8 , wherein the instructions are further executable to cause the processor to:
set a stored echo flag based on the identifying of the audio signal as a unitary voice signal; and
selectively apply the first mode or the second mode of AEC to the audio signal based on the stored echo flag.
11 . The system of claim 8 , wherein the instructions are further executable to cause the processor to receive or update the trained ML model as configured by training datasets including echo data and single-word data.
12 . The system of claim 11 , wherein:
the echo data comprises recording, clipping, distortion, and room simulation data; and
the single-word data comprises far-end interrupt echo data and near-end interrupt voice data.
13 . The system of claim 11 , wherein the trained ML model comprises:
a plurality of convolutional neural networks with node weights configured by the training datasets;
a classifier configured to identify a single-speaker class; and
a plurality of fully connected layers disposed between the plurality of convolutional neural networks and the classifier.
14 . The system of claim 8 , wherein the instructions are further executable to cause the processor to post process the unitary voice signal.
15 . A non-transitory computer-readable medium comprising code that is executable by a processor for causing the processor to:
access an audio signal;
identify, using a trained, machine-learning (“ML”) model, the audio signal as a unitary voice signal;
apply a first mode of acoustic echo cancelation (AEC) to an audio frame of the unitary voice signal to produce a test frame;
measure a residual echo in the test frame to produce a residual value;
compare the residual value to a threshold; and
apply a second mode of AEC to the audio signal based on the comparing.
16 . The non-transitory computer-readable medium of claim 15 , wherein the threshold comprises a plurality of thresholds, and the second mode of AEC comprises a plurality of modes with different attenuation levels for an echo in the audio signal.
17 . The non-transitory computer-readable medium of claim 15 , wherein the code is executable for causing the processor to receive or update the trained ML model as configured by training datasets including echo data and single-word data.
18 . The non-transitory computer-readable medium of claim 17 , wherein:
the echo data comprises recording, clipping, distortion, and room simulation data; and
the single-word data comprises far-end interrupt echo data and near-end interrupt voice data.
19 . The non-transitory computer-readable medium of claim 17 , wherein the trained ML model comprises:
a plurality of convolutional neural networks with node weights configured by the training datasets;
a classifier configured to identify a single-speaker class; and
a plurality of fully connected layers disposed between the plurality of convolutional neural networks and the classifier.
20 . The non-transitory computer-readable medium of claim 15 , wherein the code is executable for causing the processor to post process the unitary voice signal.