Method for extracting speech from degraded signals by predicting the inputs to a speech vocoder
View Patent ↗A method for Parametric resynthesis (PR) producing an audible signal. A degraded audio signal is received which includes a distorted target audio signal. A prediction model predicts parameters of the audible signal from the degraded signal. The prediction model was trained to minimize a loss function between the target audio signal and the predicted audible signal. The predicted parameters are provided to a waveform generator which synthesizes the audible signal.
1. A method for Parametric resynthesis (PR) producing a predicted audible signal from a degraded audio signal produced by distorting a target audio signal, the method comprising:
receiving the degraded audio signal which is derived from the target audio signal;
predicting, with a prediction model, a plurality of parameters of the predicted audible signal from the degraded audio signal including removing noise from the degraded audio signal to output a prediction of a clean acoustic feature including the plurality of parameters;
providing the plurality of parameters to a waveform generator; and
synthesizing the predicted audible signal with the waveform generator;
wherein the prediction model has been trained to reduce a loss function between the target audio signal and the predicted audible signal.
2. The method as recited in claim 1 , wherein the waveform generator is a vocoder.
3. The method as recited in claim 2 , wherein the vocoder is a non-neural vocoder.
4. The method as recited in claim 2 , wherein the vocoder is a neural vocoder.
5. The method as recited in claim 4 , wherein the neural vocoder is a WaveNet vocoder.
6. The method as recited in claim 4 , wherein the neural vocoder is a WaveGlow vocoder.
7. The method as recited in cl aim 4 , wherein the neural vocoder is an LPCNet vocoder.
8. The method as recited in claim 1 , wherein the plurality of parameters includes at least one of:
(1) a spectral envelope;
(2) a log fundamental frequency (F0); or
(3) an aperiodic energy of the spectral envelope.
9. The method as recited in claim 1 , wherein the plurality of parameters includes a log mel spectrum of individual frames of audio, creating a log mel spectrogram.
10. The method of claim 9 , where the loss function is a mean square error between the target audio signal and the predicted audible signal in the log mel spectrogram.
11. The method of claim 1 , where the loss function is a mean square error between the plurality of parameters of the predicted audible signal and corresponding parameters of the target audio signal.
12. The method of claim 1 , where the loss function is a mean square error between target audio signal and the predicted audible signal in a time domain.
13. The method of claim 1 , where the degraded audio signal is produced by (1) filtering the target audio signal to produce a filtered signal, adding noise to the filtered signal to produce a summed signal, and then non-linearly processing a sum of the filtered signal and the summed signal.
14. The method of claim 1 , where the loss function is a negative conditional log-likelihood of clean speech under a probabilistic vocoder given the plurality of parameters.
15. The method of claim 1 , where the loss function is a categorical cross-entropy loss of a predicted probability of an excitation of a linear prediction model.