Real-time low-complexity echo cancellation
Methods, systems, and apparatus, including computer programs encoded on computer storage media relate to a method for acoustic echo cancellation. The system inputs one or more signal representations into an acoustic echo cancellation network comprising one or more network blocks to generate a mask, each network block comprising one or more convolutional blocks, each convolutional block comprising one or more neural networks. The system combines the mask and a near-end audio signal representation to generate an echo-cancelled audio signal representation. The system generates an echo-cancelled audio signal based on the echo-cancelled audio signal representation.
1. A computer-implemented method for acoustic echo cancellation, comprising:
generating a far-end audio signal representation, a near-end audio signal representation, and a linear output signal representation based on a far-end audio signal, a near-end audio signal, and a linear output signal, respectively;
inputting the far-end audio signal representation, the near-end audio signal representation, and the linear output signal representation into an AEC network comprising one or more network blocks to generate a mask, each network block comprising one or more convolutional blocks, each convolutional block comprising one or more neural networks;
combining the mask and the near-end audio signal representation to generate an echo-cancelled audio signal representation; and
generating an echo-cancelled audio signal based on the echo-cancelled audio signal representation.
2. The method of claim 1 , wherein the far-end audio signal representation, the near-end audio signal representation, and the linear output signal representation comprise STFTs of the far-end audio signal, the near-end audio signal, and the linear output signal, respectively.
3. The method of claim 2 , wherein the echo-cancelled audio signal is generated based on an inverse STFT of the echo-cancelled audio signal representation.
4. The method of claim 1 , wherein each network block comprises a series of convolutional blocks of increasing dilation, the output of each convolutional block in the series being input to the next convolutional block in the series.
5. The method of claim 4 , further comprising:
summing the outputs of one or more convolutional blocks in a network block and inputting the sum to a next network block.
6. The method of claim 5 , further comprising:
fusing the sum of the outputs of the one or more convolutional blocks in the network block with an embedding of the far-end audio signal representation, the near-end audio signal representation, and the linear output signal representation prior to inputting the sum to the next network block.
7. The method of claim 1 , wherein the far-end audio signal comprises a first speech audio signal, the near-end audio signal comprises a second speech audio signal combined with an echo of the far-end audio signal, and the linear output signal comprises output of a DSP AEC linear filter, and the AEC network is trained by minimizing a loss function based on the difference between the echo-cancelled audio signal and the second speech audio signal.
8. A non-transitory computer readable medium that stores executable program instructions that when executed by one or more computing devices configure the one or more computing devices to:
generate a far-end audio signal representation, a near-end audio signal representation, and a linear output signal representation based on a far-end audio signal, a near-end audio signal, and a linear output signal, respectively;
input the far-end audio signal representation, the near-end audio signal representation, and the linear output signal representation into an AEC network comprising one or more network blocks to generate a mask, each network block comprising one or more convolutional blocks, each convolutional block comprising one or more neural networks;
combine the mask and the near-end audio signal representation to generate an echo-cancelled audio signal representation; and
generate an echo-cancelled audio signal based on the echo-cancelled audio signal representation.
9. The non-transitory computer readable medium of claim 8 , wherein the far-end audio signal representation, the near-end audio signal representation, and the linear output signal representation comprise STFTs of the far-end audio signal, the near-end audio signal, and the linear output signal, respectively.
10. The non-transitory computer readable medium of claim 9 , wherein the echo-cancelled audio signal is generated based on an inverse STFT of the echo-cancelled audio signal representation.
11. The non-transitory computer readable medium of claim 8 , wherein each network block comprises a series of convolutional blocks of increasing dilation, the output of each convolutional block in the series being input to the next convolutional block in the series.
12. The non-transitory computer readable medium of claim 11 , further comprising executable program instructions that when executed by one or more computing devices configure the one or more computing devices to:
sum the outputs of one or more convolutional blocks in a network block and inputting the sum to a next network block.
13. The non-transitory computer readable medium of claim 12 , further comprising executable program instructions that when executed by one or more computing devices configure the one or more computing devices to:
fuse the sum of the outputs of the one or more convolutional blocks in the network block with an embedding of the far-end audio signal representation, the near-end audio signal representation, and the linear output signal representation prior to inputting the sum to the next network block.
14. The non-transitory computer readable medium of claim 8 , wherein the far-end audio signal comprises a first speech audio signal, the near-end audio signal comprises a second speech audio signal combined with an echo of the far-end audio signal, and the linear output signal comprises output of a DSP AEC linear filter, and the AEC network is trained by minimizing a loss function based on the difference between the echo-cancelled audio signal and the second speech audio signal.
15. An acoustic echo cancellation system comprising:
a non-transitory computer-readable medium; and
one or more processors configured to execute processor-executable instructions stored in the non-transitory computer-readable medium, the processor-executable instructions configured to cause the one or more processors to:
generate a far-end audio signal representation, a near-end audio signal representation, and a linear output signal representation based on a far-end audio signal, a near-end audio signal, and a linear output signal, respectively;
input the far-end audio signal representation, the near-end audio signal representation, and the linear output signal representation into an acoustic echo cancellation (AEC) network comprising one or more network blocks to generate a mask, each network block comprising one or more convolutional blocks, each convolutional block comprising one or more neural networks;
combine the mask and the near-end audio signal representation to generate an echo-cancelled audio signal representation; and
generate an echo-cancelled audio signal based on the echo-cancelled audio signal representation.
16. The system of claim 15 , wherein the far-end audio signal representation, the near-end audio signal representation, and the linear output signal representation comprise Short-time Fourier Transforms (STFT) of the far-end audio signal, the near-end audio signal, and the linear output signal, respectively.
17. The system of claim 16 , wherein the echo-cancelled audio signal is generated based on an inverse STFT of the echo-cancelled audio signal representation.
18. The system of claim 15 , wherein each network block comprises a series of convolutional blocks of increasing dilation, the output of each convolutional block in the series being input to the next convolutional block in the series.
19. The system of claim 18 , wherein the one or more processors are further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:
sum the outputs of one or more convolutional blocks in a network block and inputting the sum to a next network block.
20. The system of claim 19 , wherein the one or more processors are further configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:
fuse the sum of the outputs of the one or more convolutional blocks in the network block with an embedding of the far-end audio signal representation, the near-end audio signal representation, and the linear output signal representation prior to inputting the sum to the next network block.