GAN image generation from feature regularization
Systems and methods for training a Generative Adversarial Network (GAN) using feature regularization are described herein. Embodiments are configured to generate a candidate image using a generator network of a GAN, classify the candidate image as real or generated using a discriminator network of the GAN, and train the GAN to generate realistic images based on the classifying of the candidate image. The training process includes regularizing a gradient with respect to features extracted using a discriminator network of the GAN.
1 . A method comprising:
obtaining an input vector; and
generating an image based on the input vector using a generative adversarial network (GAN) that includes at least one encoder layer and at least one classifier layer, wherein the at least one classifier layer is trained by computing a regularization loss that includes a gradient with respect to encoded features, wherein the at least one encoder layer generates the encoded features and the at least one classifier layer generates a classification output based on the encoded features, and wherein the gradient has a same dimension as the encoded features.
2 . The method of claim 1 , wherein:
the gradient is computed based on the classification output independently of the at least one encoder layer.
3 . The method of claim 1 , wherein:
the input comprises a text prompt.
4 . The method of claim 1 , wherein:
the image comprises a face image, and a discriminator network of the GAN is trained to classify face images as real or synthetic.
5 . An apparatus comprising:
at least one processor;
at least one memory including instructions executable by the processor; and
the apparatus further comprising a GAN comprising parameters stored in the at least one memory, wherein the GAN includes at least one encoder layer and at least one classifier layer, wherein the at least one classifier is trained to generate images by computing a regularization loss that incudes a gradient with respect to encoded features generated by the GAN, wherein the at least one encoder layer generates the encoded features and the at least one classifier layer generates a classification output based on the encoded features, and wherein the gradient has a same dimension as the encoded features.
6 . The apparatus of claim 5 , wherein:
the GAN comprises a generator network configured to generate the images.
7 . The apparatus of claim 5 , further comprising:
a training component configured to compute a regularization loss, wherein the training is based on the regularization loss.
8 . The apparatus of claim 7 , wherein:
the regularization loss comprises an R1 regularization loss.
9 . The apparatus of claim 5 , wherein:
the GAN comprises a discriminator network configured to classify the images, wherein the training is based on the classifying of the images.
10 . The apparatus of claim 9 , wherein:
the discriminator network of the GAN comprises a pretrained encoder.
11 . The apparatus of claim 9 , wherein:
the discriminator network of the GAN comprises a plurality of classifiers.
12 . The apparatus of claim 11 , wherein:
the discriminator network of the GAN comprises a plurality of encoders corresponding to the plurality of classifiers, respectively.
13 . A non-transitory computer readable medium storing code, the code comprising instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
obtaining an input vector; and
generating an image based on the input vector using a generative adversarial network (GAN) that includes at least one encoder layer and at least one classifier layer, wherein the at least one classifier layer is trained by computing a regularization loss that includes a gradient with respect to encoded features, wherein the at least one encoder layer generates the encoded features and the at least one classifier layer generates a classification output based on the encoded features, and wherein the gradient has a same dimension as the encoded features.
14 . The non-transitory computer readable medium of claim 13 , wherein:
the gradient is computed based on the classification output independently of the at least one encoder layer.
15 . The non-transitory computer readable medium of claim 13 , wherein:
the input comprises a text prompt.
16 . The non-transitory computer readable medium of claim 13 , wherein:
the image comprises a face image, and a discriminator network of the GAN is trained to classify face images as real or synthetic.
17 . The non-transitory computer readable medium of claim 16 , wherein:
the discriminator network of the GAN comprises a pretrained encoder.
18 . The non-transitory computer readable medium of claim 13 , wherein:
the discriminator network of the GAN comprises a plurality of classifiers.
19 . The non-transitory computer readable medium of claim 18 , wherein:
the discriminator network of the GAN comprises a plurality of encoders corresponding to the plurality of classifiers, respectively.
20 . The non-transitory computer readable medium of claim 13 , the code further comprising instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
computing a regularization loss, wherein the regularization loss comprises an R1 regularization loss.