Method and a server for generating a waveform

1. A method of generating a waveform based on a spectrogram and a noise input, the spectrogram having been generated based on a text, the waveform being a digital audio representation of the text, the method being executable by a server, the method comprising:

acquiring, by the server, a trained flow-based vocoder including a plurality of invertible blocks, the trained flow-based vocoder having been trained to generate the waveform based on the spectrogram and the noise input;

acquiring, by the server, an untrained feed-forward vocoder including a plurality of non-invertible blocks,

the trained flow-based vocoder and the untrained feed-forward vocoder forming a student-teacher network;

executing a training process in the student-teacher network, during a given training iteration of the training process:

generating, by the server, a teacher waveform by the trained flow-based vocoder using a first spectrogram and a first noise input;

generating, by the server, a student waveform by the untrained feed-forward vocoder using the first spectrogram and the first noise input;

generating, by the server, a loss value for the given training iteration, wherein generating the loss value comprises:

determining a reconstruction-based value based on a difference between the teacher waveform and the student waveform,

determining a feature-based loss value based on a difference between features of the teacher waveform and features of the student waveform, and

determining the loss value based on the reconstruction-based loss value and the feature-based loss value; and

training, by the server, the untrained feed-forward vocoder to generate the waveform using the loss value for approximating a mapping between the first noise input and the teacher waveform of the flow-based vocoder;

acquiring, by the server, the spectrogram and the noise input; and

using, by the server, the trained feed-forward vocoder in lieu of the trained flow-based vocoder for generating the waveform based on the spectrogram and the noise input.

2. The method of claim 1 , wherein the spectrogram is a mel-scaled spectrogram.

3. The method of claim 1 , wherein the noise input is representative of a Gaussian distribution of noise values.

4. The method of claim 1 , wherein the method further comprises storing, by the server, the trained feed-forward vocoder in a storage.

5. The method of claim 1 , wherein the trained flow-based vocoder is a conditional normalizing-flow-based vocoder.

6. The method of claim 1 , wherein the trained feed-forward vocoder is a Convolutional Neural Network (CNN) based vocoder.

7. A method of generating an output using a conditioning input and a noise input, the method executable by a server, the method comprising:

acquiring, by the server, a trained conditional flow-based model including a plurality of invertible blocks, the trained conditional flow-based model having been trained to generate the output based on the conditioning input and the noise input;

acquiring, by the server, an untrained feed-forward model including a plurality of non-invertible blocks,

the trained conditional flow-based model and the untrained feed-forward model forming a student-teacher network;

executing a training process in the student-teacher network, during a given training iteration of the training process:

generating, by the server, a teacher waveform by the trained conditional flow-based model using a first conditioning input and a first noise input,

generating, by the server, a student waveform by the untrained feed-forward model using the first conditioning input and the first noise input,

generating, by the server, a loss value for the given training iteration, wherein generating the loss value comprises:

determining a reconstruction-based loss value based on a difference between the teacher waveform and the student waveform;

determining a feature-based loss value based on a difference between features of the teacher waveform and features of the student waveform; and

determining the loss value based on the reconstruction-based loss value and the feature-based loss value, and

training, by the server, the untrained feed-forward model by using the loss value for approximating a mapping between the first noise input and the teacher output of the conditional flow-based model;

acquiring, by the server, the conditioning input and the noise input; and

using, by the server, the trained feed-forward model in lieu of the trained conditional flow-based model for generating the output based on the conditioning input and the noise input.

8. The method of claim 7 , wherein the trained conditional flow-based model is a trained conditional flow-based vocoder, the trained feed-forward model is a trained feed-forward vocoder, the conditioning input is a spectrogram, and the output is a waveform.

9. A server for generating a waveform based on a spectrogram and a noise input, the spectrogram having been generated based on a text, the waveform being a digital audio representation of the text, the server comprising at least one processor and at least one memory storing executable instructions which, when executed by the at least one processor cause the server to:

acquire a trained flow-based vocoder including a plurality of invertible blocks, the trained flow-based vocoder having been trained to generate the waveform based on the spectrogram and the noise input;

acquire an untrained feed-forward vocoder including a plurality of non-invertible blocks,

the trained flow-based vocoder and the untrained feed-forward vocoder forming a student-teacher network;

execute a training process in the student-teacher network, during a given training iteration of the training process comprises:

generating a teacher waveform by the trained flow-based vocoder using a first spectrogram and a first noise input;

generating a student waveform by the untrained feed-forward vocoder using the first spectrogram and the first noise input;

generating a loss value for the given training iteration, wherein generating the loss value comprises:

determining a reconstruction-based loss value based on a difference between the teacher waveform and the student waveform,

determining a feature-based loss value based on a difference between features of the teacher waveform and features of the student waveform, and

determining the loss value based on the reconstruction-based loss value and the feature-based loss value; and

training the untrained feed-forward vocoder to generate the waveform using the loss value for approximating a mapping between the first noise input and the teacher waveform of the flow-based vocoder;

acquire the spectrogram and the noise input; and

use the trained feed-forward vocoder in lieu of the trained flow-based vocoder for generating the waveform based on the spectrogram and the noise input.

10. The server of claim 9 , wherein the spectrogram is a mel-scaled spectrogram.

11. The server of claim 9 , wherein the noise input is representative of a Gaussian distribution of noise values.

12. The server of claim 9 , wherein the instructions further cause the server to store the trained feed-forward vocoder in a storage.

13. The server of claim 9 , wherein the trained flow-based vocoder is a conditional normalizing-flow-based vocoder.

14. The server of claim 9 , wherein the trained feed-forward vocoder is a Convolutional Neural Network (CNN) based vocoder.