Systems and methods for training artificial neural networks
A system including: one or more processors; and memory including instructions that, when executed by the one or more processors, cause the one or more processors to: generate augmented input data by mixing noise components of training data; train a first neural network based on the augmented input data and ground truth data of the training data to output a first prediction of clean speech; lock trainable parameters of the first neural network as a result of the training of the first neural network; and train a second neural network according to the augmented input data and predictions of the first neural network to output a second prediction of the clean speech.
1 . A system comprising:
one or more processors; and
memory comprising instructions that, when executed by the one or more processors, cause the one or more processors to:
generate augmented input data by mixing noise components corresponding to at least two noise spectrograms, each of the at least two noise spectrograms being associated with its corresponding clean spectrogram of samples of training data, and adding the mixed noise components to a clean component corresponding to a clean spectrogram of a sample of the training data;
train a first neural network based on the augmented input data and ground truth data of the training data to output a first prediction of clean speech;
lock trainable parameters of the first neural network as a result of the training of the first neural network; and
train a second neural network according to the augmented input data and predictions of the first neural network to output a second prediction of the clean speech.
2 . The system of claim 1 , wherein a number of trainable parameters in the second neural network is smaller than a number of the trainable parameters in the first neural network.
3 . The system of claim 1 , wherein the first neural network is trained according to a first loss function and a second loss function.
4 . The system of claim 3 , wherein the second loss function is based on the first loss function and a hyper-parameter that defines an extent of the mixing of the noise components of the training data.
5 . The system of claim 3 , wherein the second neural network is trained according to a third loss function different from the first and second loss functions.
6 . The system of claim 1 , wherein to train the second neural network, the instructions further cause the one or more processors to mix intermediate features of input data at a hidden layer of each of the first and second neural networks to generate a batch of mixed intermediate features of the input data for each of the first and second neural networks.
7 . The system of claim 6 , wherein to train the second neural network, the instructions further cause the one or more processors to continue a forward pass from the hidden layer of each of the first and second neural networks to a corresponding output of each of the first and second neural networks using the batch of mixed intermediate features.
8 . The system of claim 7 , wherein to train the second neural network, the instructions further cause the one or more processors to:
calculate a loss between the corresponding output of the second neural network and the corresponding output of the first neural network, based on the mixed batch of intermediate features; and
adjust trainable parameters of the second neural network based on the loss.
9 . A method comprising:
generating, by one or more processors, augmented input data by mixing noise components corresponding to at least two noise spectrograms, each of the at least two noise spectrograms being associated with its corresponding clean spectrogram of samples of training data, and adding the mixed noise components to a clean component corresponding to a clean spectrogram of a sample of the training data;
training, by the one or more processors, a first neural network based on the augmented input data and ground truth data of the training data to output a first prediction of clean speech;
locking, by the one or more processors, trainable parameters of the first neural network as a result of the training of the first neural network; and
training, by the one or more processors, a second neural network according to the augmented input data and predictions of the first neural network to output a second prediction of the clean speech.
10 . The method of claim 9 , wherein a number of trainable parameters in the second neural network is smaller than a number of the trainable parameters in the first neural network.
11 . The method of claim 9 , wherein the first neural network is trained according to a first loss function and a second loss function.
12 . The method of claim 11 , wherein the second loss function is based on the first loss function and a hyper-parameter that defines an extent of the mixing of the noise components of the training data.
13 . The method of claim 11 , wherein the second neural network is trained according to a third loss function different from the first and second loss functions.
14 . The method of claim 9 , wherein to train the second neural network, the method further comprises mixing, by the one or more processors, intermediate features of input data at a hidden layer of each of the first and second neural networks to generate a batch of mixed intermediate features of the input data for each of the first and second neural networks.
15 . The method of claim 14 , wherein to train the second neural network, the method further comprises continuing, by the one or more processors, a forward pass from the hidden layer of each of the first and second neural networks to a corresponding output of each of the first and second neural networks using the batch of mixed intermediate features.
16 . The method of claim 15 , wherein to train the second neural network, the method further comprises:
calculating, by the one or more processors, a loss between the corresponding output of the second neural network and the corresponding output of the first neural network, based on the mixed batch of intermediate features; and
adjusting, by the one or more processors, trainable parameters of the second neural network based on the loss.
17 . A computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising:
generating augmented input data by mixing noise components corresponding to at least two noise spectrograms, each of the at least two noise spectrograms being associated with its corresponding clean spectrogram of samples of training data, and adding the mixed noise components to a clean component corresponding to a clean spectrogram of a sample of the training data;
training a first neural network based on the augmented input data and ground truth data of the training data to output a first prediction of clean speech;
locking trainable parameters of the first neural network as a result of the training of the first neural network; and
training a second neural network according to the augmented input data and predictions of the first neural network to output a second prediction of the clean speech.
18 . The computer-readable medium of claim 17 , wherein to train the second neural network, the instructions further cause the one or more processors to perform the method comprising mixing the augmented input data in a hidden layer of each of the first and second neural networks to generate a mixed batch of intermediate features of the augmented input data.
19 . The computer-readable medium of claim 18 , wherein to train the second neural network, the instructions further cause the one or more processors to perform the method comprising continuing a forward pass from the hidden layer of each of the first and second neural networks to a corresponding output of each of the first and second neural networks using the mixed batch of intermediate features.
20 . The computer-readable medium of claim 19 , wherein to train the second neural network, the instructions further cause the one or more processors to perform the method comprising:
calculating a loss between the corresponding output of the second neural network and the corresponding output of the first neural network, based on the mixed batch of intermediate features; and
adjusting trainable parameters of the second neural network based on the loss.