Per frame input estimation for post filter echo suppression

1 . A system, comprising:

a microphone;

a speaker

at least one processor; and

a memory, storing program instructions that when executed by the at least one processor, cause the at least one processor to:

receive captured audio data of a two-way communication via the microphone in a location that includes the speaker that plays far-end audio data of the two-way communication;

generate enhanced audio data from the captured audio data to suppress echo caused by inclusion of the far-end audio data in the captured audio data, wherein to generate the enhanced audio data, the program instructions cause the at least one processor to:

respectively process individual frames of the captured audio data in a sub-band domain as a plurality of different sub-bands, wherein to respectively process the individual frames, the program instructions cause the at least one processor to:

apply a machine learning model, trained to:

 accept, as inputs, the plurality of different sub-bands of a frame of the captured audio data and corresponding sub-bands of a corresponding frame of the far-end audio data; and

 generate, as outputs, post-filter inputs for the frame of the captured audio data, wherein the post filter inputs comprise an attenuation parameter, a near-end estimate, and echo estimate;

apply a post-filter to the plurality of different sub-bands of the frame of the captured audio data according to the attenuation parameter and a ratio of the near-end estimate to the echo estimate to generate an enhanced frame to include as part of the enhanced audio data; and

send the enhanced audio data to a destination.

2 . The system of claim 1 , wherein the attenuation parameter output for a first frame of the individual frames is different than the attenuation parameter output for a second frame of the individual frames.

3 . The system of claim 1 , wherein the machine learning model is a neural network comprising at least two paths, wherein one of the at least two paths is an echo path used to determine the echo estimate.

4 . The system of claim 1 , wherein the enhanced audio data is sent to the destination via a communication service implemented as part of a provider network.

5 . A method, comprising:

obtaining captured audio data of a two-way communication via a microphone in a location that includes a speaker that plays far-end audio data of the two-way communication;

for individual frames of the captured audio data that are respectively processed as a plurality of different sub-bands:

applying a machine learning model, trained to:

accept, as inputs, the plurality of different sub-bands of a frame of the captured audio data and corresponding sub-bands of a corresponding frame of the far-end audio data; and

generate, as outputs, post-filter inputs for the frame of the captured audio data, wherein the post filter inputs comprise an attenuation parameter, a near-end estimate, and echo estimate;

applying a post-filter to the plurality of different sub-bands of the frame of the captured audio data according to the attenuation parameter and a ratio of the near-end estimate to the echo estimate to remove echo data caused by the far-end audio data in the captured audio data to generate an enhanced frame of the captured audio data; and

providing the generated enhanced frames of the captured audio data.

6 . The method of claim 5 , wherein the attenuation parameter output for a first frame of the individual frames is different than the attenuation parameter output for a second frame of the individual frames.

7 . The method of claim 5 , wherein the machine learning model is a neural network comprising at least two paths and wherein one of the at least two paths is an echo path used to determine the echo estimate.

8 . The method of claim 5 , wherein the attenuation parameter is provided as the post-filter input to the post-filter in a Bark scale.

9 . The method of claim 5 , wherein the individual frames are determined according to a window of time and wherein the window of time indicates a rate which the post-filter inputs are generated for the captured audio data.

10 . The method of claim 5 , wherein the plurality of different sub-bands of the frame of the captured audio data and the corresponding sub-bands of the corresponding frame of the far-end audio data are provided as inputs to the machine learning model in Bark scales.

11 . The method of claim 5 , further comprising training the machine learning model using a total loss function that combines a coupling loss, echo loss, scaled-echo loss, near-end loss, scaled near-end loss, and microphone loss, the total loss function being applied using a supervised learning technique with respect to a training data set comprising clean speech, noisy speech and music.

12 . The method of claim 5 , wherein the obtaining the captured audio data, the applying the machine learning model, the applying the post filter, and the providing the enhanced audio frames is performed by a communication service implemented as part of a provider network.

13 . The method of claim 5 , wherein the generated enhanced audio frames are provided to a destination via a communication service implemented as part of a provider network.

14 . One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices cause the one or more computing devices to implement:

receiving captured audio data of a two-way communication via a microphone in a location that includes a speaker that plays far-end audio data of the two-way communication;

for individual frames of the captured audio data that are respectively processed as a plurality of different sub-bands:

applying a machine learning model, trained to:

accept, as inputs, the plurality of different sub-bands of a frame of the captured audio data and corresponding sub-bands of a corresponding frame of the far-end audio data; and

generate, as outputs, post-filter inputs for the frame of the captured audio data, wherein the post filter inputs comprise an attenuation parameter, a near-end estimate, and echo estimate;

applying a post-filter to the plurality of different sub-bands of the frame of the captured audio data according to the attenuation parameter and a ratio of the near-end estimate to the echo estimate remove echo data caused by the far-end audio data in the captured audio data to generate an enhanced frame of the captured audio data; and

sending the generated enhanced frames of the captured audio data to a destination.

15 . The one or more non-transitory, computer-readable storage media of claim 14 , wherein the attenuation parameter output for a first frame of the individual frames is different than the attenuation parameter output for a second frame of the individual frames.

16 . The one or more non-transitory, computer-readable storage media of claim 14 , wherein the machine learning model is a neural network comprising at least two paths, wherein one of the at least two paths is an echo path used to determine the echo estimate.

17 . The one or more non-transitory, computer-readable storage media of claim 14 , wherein the attenuation parameter is provided as the post-filter input to the post-filter in a Bark scale.

18 . The one or more non-transitory, computer-readable storage media of claim 14 , wherein the individual frames are determined according to a window of time and wherein the window of time indicates a rate which the post-filter inputs are generated for the captured audio data.

19 . The one or more non-transitory, computer-readable storage media of claim 14 , wherein the plurality of different sub-bands of the frame of the captured audio data and the corresponding sub-bands of the corresponding frame of the far-end audio data are provided as inputs to the machine learning model in Bark scales.

20 . The one or more non-transitory, computer-readable storage media of claim 14 , wherein the one or more computing devices is a communication device that is a participant in the two way communication that includes the microphone and the speaker.