Method and apparatus for packet loss concealment using generative adversarial network
View Patent ↗Disclosed are a packet loss concealment method and apparatus a using a generative adversarial network. A method for packet loss concealment in voice communication may include training a classification model based on a generative adversarial network (GAN) with respect to a voice signal including a plurality of frames, training a generative model having a contention relation with the classification model based on the GAN, estimating lost packet information based on the trained generative model with respect to the voice signal encoded by a codec, and restoring a lost packet based on the estimated packet information.
1. A method for packet loss concealment in voice communication, the method comprising:
training a classification model based on a generative adversarial network (GAN) with respect to a voice signal comprising a plurality of frames;
training a generative model having a contention relation with the classification model based on the GAN;
estimating lost packet information based on the trained generative model with respect to the voice signal encoded by a codec; and
restoring a lost packet based on the estimated packet information,
wherein estimating the lost packet information comprises:
setting a specific number of previous frames predetermined prior to a target frame comprising a lost packet from among the plurality of frames as an input to the generative model,
setting the target frame as an output of the generative model, and
estimating the lost packet information.
2. The method of claim 1 , wherein the classification model determines whether data corresponding to an input signal is data corresponding to a real voice signal or data corresponding to a voice signal generated through the generative model.
3. The method of claim 1 , wherein training the generative model comprises training the generative model to classify data generated through the generative model as data corresponding to a real voice signal in the classification model.
4. The method of claim 3 , wherein training the classification model comprises training the classification model to classify data generated through the generative model as fake data not data corresponding to a real voice signal through the training.
5. The method of claim 1 , wherein training the classification model comprises training the classification model to classify data corresponding to a frame before a packet is lost as real data and classify data of a lost packet estimated based on the generative model as fake data with respect to the voice signal.
6. The method of claim 1 , wherein the classification model and the generative model are trained based on a convolutional neural network.
7. A method for packet loss concealment in voice communication, the method comprising:
training a classification model based on a generative adversarial network (GAN) with respect to a voice signal comprising a plurality of frames;
training a generative model having a contention relation with the classification model based on the GAN;
estimating lost packet information based on the trained generative model with respect to the voice signal encoded by a codec; and
restoring a lost packet based on the estimated packet information,
wherein the lost packet is restored based on a fast Fourier transform (FFT) coefficient and phase extracted from each of a specific number of previous frames predetermined prior to a target frame comprising a lost packet from among the plurality of frames and the trained generative model.
8. An apparatus for packet loss concealment in voice communication, the apparatus comprising:
at least a processor configured to:
train a classification model based on a generative adversarial network (GAN) with respect to a voice signal comprising a plurality of frames by a classification model training unit;
train a generative model having a contention relation with the classification model based on the GAN by a generative model training unit; and
estimate lost packet information based on the trained generative model with respect to the voice signal encoded by a codec and restore a lost packet based on the estimated packet information,
wherein estimating the lost packet information comprises:
setting a specific number of previous frames predetermined prior to a target frame comprising a lost packet from among the plurality of frames as an input to the generative model,
setting the target frame as an output of the generative model, and
estimating the lost packet information.
9. The apparatus of claim 8 , wherein the classification model determines whether data corresponding to an input signal is data corresponding to a real voice signal or data corresponding to a voice signal generated through the generative model.
10. The apparatus of claim 8 , wherein the generative model training unit trains the generative model to classify data generated through the generative model as data corresponding to a real voice signal in the classification model.
11. The apparatus of claim 10 , wherein the generative model training unit trains the classification model to classify data generated through the generative model as fake data not data corresponding to a real voice signal through the training.
12. The apparatus of claim 8 , wherein the classification model training unit trains the classification model to classify data corresponding to a frame before a packet is lost as real data and to classify data of a lost packet estimated based on the generative model as fake data with respect to the voice signal.
13. The apparatus of claim 8 , wherein the classification model and the generative model are trained based on a convolutional neural network.
14. The apparatus of claim 8 , wherein the lost packet is restored based on a fast Fourier transform (FFT) coefficient and phase extracted from each of a specific number of previous frames predetermined prior to a target frame comprising a lost packet from among the plurality of frames and the trained generative model.