Method and apparatus for data augmentation
View Patent ↗Disclosed herein is a method for data augmentation, which includes pretraining latent variables using first data corresponding to target speech and second data corresponding to general speech, training data augmentation parameters by receiving the first data and the second data as input, and augmenting target data using the first data and the second data through the pretrained latent variables and the trained parameters.
1. A method for data augmentation by an apparatus, the apparatus including a processor and a memory operably coupled to the processor, wherein the memory stores program instructions to be executed by the processor, the method comprising:
pretraining latent variables using first data corresponding to target speech and second data corresponding to general speech;
training data augmentation parameters by receiving the first data and the second data as input; and
augmenting target data using the first data and the second data through the pretrained latent variables and the trained parameters,
wherein the latent variables include
a first latent variable corresponding to content attributes including a phone sequence of refined speech corpus of speech data; and
a second latent variable corresponding to environment information of speech data,
wherein the environment information of the speech data includes information about at least one of a channel, noise, an accent, a tone, a rhythm, and a speech tempo of a speaker.
2. The method of claim 1 , wherein pretraining the latent variables includes
inferring, by an encoder of a variational autoencoder, latent variables using the first data and the second data; and
performing, by a decoder of the variational autoencoder, training so as to generate input data by receiving the latent variables as input, and
wherein training the data augmentation parameters comprises performing training using a structure in which a generative adversarial network is added to the variational autoencoder.
3. The method of claim 2 , wherein training the data augmentation parameters includes
generating first output through the variational autoencoder by receiving the first data as input;
generating second output through the decoder by receiving the second latent variable corresponding to the first data and the first latent variable corresponding to the second data; and
differentiating, by a discriminator, the first output and the second output from each other.
4. The method of claim 3 , wherein augmenting the target data comprises augmenting the target data through the decoder by receiving the second latent variable corresponding to the first data and the first latent variable corresponding to the second data.
5. The method of claim 4 , wherein training the data augmentation parameters comprises training the parameters using a first loss function corresponding to the encoder, a second loss function corresponding to the decoder, and a third loss function corresponding to the discriminator.
6. The method of claim 5 , wherein the parameters include a first parameter corresponding to the encoder, a second parameter corresponding to the decoder, and a third parameter corresponding to the discriminator.
7. An apparatus for data augmentation, comprising:
one or more processors; and
executable memory for storing at least one program executed by the one or more processors,
wherein the at least one program is configured to
pretrain latent variables using first data corresponding to target speech and second data corresponding to general speech;
train data augmentation parameters by receiving the first data and the second data as input; and
augment target data using the first data and the second data through the pretrained latent variables and the trained parameters,
wherein the latent variables include
a first latent variable corresponding to content attributes including a phone sequence of refined speech corpus of speech data; and
a second latent variable corresponding to environment information of speech data,
wherein the environment information of the speech data includes information about at least one of a channel, noise, an accent, a tone, a rhythm, and a speech tempo of a speaker.
8. The apparatus of claim 7 , wherein the at least one program performs training such that an encoder of a variational autoencoder infers latent variables using the first data and the second data and such that a decoder of the variational autoencoder generates input data by receiving the latent variables as input, and
wherein the at least one program trains the parameters using a structure in which a generative adversarial network is added to the variational autoencoder.
9. The apparatus of claim 8 , wherein the at least one program trains the parameters by generating first output through the variational autoencoder receiving the first data as input, generating second output through the decoder receiving the second latent variable corresponding to the first data and the first latent variable corresponding to the second data, and making a discriminator differentiate between the first output and the second output.
10. The apparatus of claim 9 , wherein the at least one program augments the target data through the decoder by receiving the second latent variable corresponding to the first data and the first latent variable corresponding to the second data as input.
11. The apparatus of claim 10 , wherein the at least one program trains the parameters using a first loss function corresponding to the encoder, a second loss function corresponding to the decoder, and a third loss function corresponding to the discriminator.
12. The apparatus of claim 11 , wherein the parameters include a first parameter corresponding to the encoder, a second parameter corresponding to the decoder, and a third parameter corresponding to the discriminator.