Hybrid AHS: a hybrid of kalman filter and deep learning for acoustic howling suppression
Method, apparatus, and non-transitory storage medium for hybrid acoustic howling suppression based on a frequency filter model and a deep neural network are provided. The method may include receiving a speech signal, the speech signal including target speech, feedback, and noise, and inputting the speech signal into a trained hybrid neural-network based howling suppression model, wherein the trained hybrid neural-network based howling suppression model is trained using training speech signal and pre-processed acoustic feedback from a first frequency filter model. The method may also include generating an enhanced speech signal with suppressed howling as an output of the trained hybrid neural-network based howling suppression model, wherein the enhanced speech signal is used to update parameters of the first frequency filter model.
1 . A method of hybrid acoustic howling suppression based on a frequency filter model and a deep neural network, the method being executed by at least one processor, the method comprising:
receiving a speech signal, the speech signal including target speech, feedback, and noise;
inputting the speech signal into a trained hybrid neural-network based howling suppression model, wherein the trained hybrid neural-network based howling suppression model is trained using training speech signal and pre-processed acoustic feedback from a first frequency filter model; and
generating an enhanced speech signal with suppressed howling as an output of the trained hybrid neural-network based howling suppression model, wherein the enhanced speech signal is used to update parameters of the first frequency filter model.
2 . The method of claim 1 , wherein training the hybrid neural-network based howling suppression model comprises:
generating a teacher speech signal, the teacher speech signal comprising a modified microphone signal, wherein the modified microphone signal comprises a target speech signal, a training noise signal, and a one-time playback signal, wherein the one-time playback signal is based on the target speech signal, and wherein the one-time playback signal replaces feedback in an initial microphone signal; and
training the hybrid neural-network based howling suppression model for speech separation using the teacher speech signal and the pre-processed acoustic feedback from the first frequency filter model.
3 . The method of claim 2 , wherein training the hybrid neural-network based howling suppression model for speech separation is based on a combined loss function, the combined loss function comprising a first component based on scale-invariance signal-to-distortion ratio and a second component based on a mean absolute error of spectrum magnitude in a frequency domain.
4 . The method of claim 2 , wherein training the hybrid neural-network based howling suppression model for speech separation comprises:
generating at least two reference signals, the at least two reference signals comprising a first intermediate signal based on the teacher speech signal and a second intermediate signal based on the pre-processed acoustic feedback from the first frequency filter model; and
training the hybrid neural-network based howling suppression model for speech separation using the teacher speech signal, the pre-processed acoustic feedback from the first frequency filter model, the first intermediate signal, and the second intermediate signal.
5 . The method of claim 2 , wherein the pre-processed acoustic feedback from the first frequency filter model is used only for training the hybrid neural-network based howling suppression model.
6 . The method of claim 2 , wherein the pre-processed acoustic feedback from the first frequency filter model is not used for generating the enhanced speech signal with suppressed howling as the output of the trained hybrid neural-network based howling suppression model.
7 . The method of claim 1 , wherein the trained hybrid neural-network based howling suppression model is trained in an offline manner.
8 . The method of claim 1 , wherein the first frequency filter model is based on a Kalman Filter.
9 . An apparatus for hybrid acoustic howling suppression based on a frequency filter model and a deep neural network, the apparatus comprising:
at least one memory configured to store program code; and
at least one processor configured to read the program code and operate as instructed by the program code, the program code including:
first receiving code configured to cause the at least one processor to receive a speech signal, the speech signal comprising target speech, feedback, and noise;
first inputting code configured to cause the at least one processor to input the speech signal into a trained hybrid neural-network based howling suppression model, wherein the trained hybrid neural-network based howling suppression model is trained using training speech signal and pre-processed acoustic feedback from a first frequency filter model; and
first generating code configured to cause the at least one processor to generate an enhanced speech signal with suppressed howling as an output of the trained hybrid neural-network based howling suppression model, wherein the enhanced speech signal is used to update parameters of the first frequency filter model.
10 . The apparatus of claim 9 , wherein training the hybrid neural-network based howling suppression model comprises:
second generating code configured to cause the at least one processor to generate a teacher speech signal, the teacher speech signal comprising a modified microphone signal, wherein the modified microphone signal comprises a target speech signal, a training noise signal, and a one-time playback signal, wherein the one-time playback signal is based on the target speech signal, and wherein the one-time playback signal replaces feedback in an initial microphone signal; and
first training code configured to cause the at least one processor to train the hybrid neural-network based howling suppression model for speech separation using the teacher speech signal and the pre-processed acoustic feedback from the first frequency filter model.
11 . The apparatus of claim 10 , wherein training the hybrid neural-network based howling suppression model for speech separation is based on a combined loss function, the combined loss function comprising a first component based on scale-invariance signal-to-distortion ratio and a second component based on a mean absolute error of spectrum magnitude in a frequency domain.
12 . The apparatus of claim 10 , wherein training the hybrid neural-network based howling suppression model for speech separation is based on a combined loss function, the combined loss function comprising a first component based on scale-invariance signal-to-distortion ratio and a second component based on a mean absolute error of spectrum magnitude in a frequency domain.
13 . The apparatus of claim 10 , wherein training the hybrid neural-network based howling suppression model for speech separation comprises:
third generating code configured to cause the at least one processor to generate at least two reference signals, the at least two reference signals comprising a first intermediate signal based on the teacher speech signal and a second intermediate signal based on the pre-processed acoustic feedback from the first frequency filter model; and
second training code configured to cause the at least one processor to train the hybrid neural-network based howling suppression model for speech separation using the teacher speech signal, the pre-processed acoustic feedback from the first frequency filter model, the first intermediate signal, and the second intermediate signal.
14 . The apparatus of claim 10 , wherein the pre-processed acoustic feedback from the first frequency filter model is used only for training the hybrid neural-network based howling suppression model.
15 . The apparatus of claim 10 , wherein the pre-processed acoustic feedback from the first frequency filter model is not used for generating the enhanced speech signal with suppressed howling as the output of the trained hybrid neural-network based howling suppression model.
16 . A non-transitory computer-readable medium storing instructions, the instructions comprising: one or more instructions that, when executed by one or more processors of a device for hybrid acoustic howling suppression based on a frequency filter model and a deep neural network, cause the one or more processors to:
receive a speech signal, the speech signal comprising target speech, feedback, and noise;
input the speech signal into a trained hybrid neural-network based howling suppression model, wherein the trained hybrid neural-network based howling suppression model is trained using training speech signal and pre-processed acoustic feedback from a first frequency filter model; and
generate an enhanced speech signal with suppressed howling as an output of the trained hybrid neural-network based howling suppression model, wherein the enhanced speech signal is used to update parameters of the first frequency filter model.
17 . The non-transitory computer-readable medium of claim 16 , wherein training the hybrid neural-network based howling suppression model comprises:
generating a teacher speech signal, the teacher speech signal comprising a modified microphone signal, wherein the modified microphone signal comprises a target speech signal, a training noise signal, and a one-time playback signal, wherein the one-time playback signal is based on the target speech signal, and wherein the one-time playback signal replaces feedback in an initial microphone signal; and
training the hybrid neural-network based howling suppression model for speech separation using the teacher speech signal and the pre-processed acoustic feedback from the first frequency filter model.
18 . The non-transitory computer-readable medium of claim 17 , wherein training the hybrid neural-network based howling suppression model for speech separation is based on a combined loss function, the combined loss function comprising a first component based on scale-invariance signal-to-distortion ratio and a second component based on a mean absolute error of spectrum magnitude in a frequency domain.
19 . The non-transitory computer-readable medium of claim 17 , wherein training the hybrid neural-network based howling suppression model for speech separation comprises:
generating at least two reference signals, the at least two reference signals comprising a first intermediate signal based on the teacher speech signal and a second intermediate signal based on the pre-processed acoustic feedback from the first frequency filter model; and
training the hybrid neural-network based howling suppression model for speech separation using the teacher speech signal, the pre-processed acoustic feedback from the first frequency filter model, the first intermediate signal, and the second intermediate signal.
20 . The non-transitory computer-readable medium of claim 17 , wherein the pre-processed acoustic feedback from the first frequency filter model is not used for generating the enhanced speech signal with suppressed howling as the output of the trained hybrid neural-network based howling suppression model.