Agilegan-based refinement method and framework for consistent texture generation

1. A method for generating a texturized image, the method comprising:

receiving a plurality of exemplar stylistic images;

training a first generative adversarial network (GAN) generator using transfer learning based on the received plurality of exemplar stylistic images;

receiving a plurality of training images;

training an encoder using the plurality of training images and another second GAN generator, the encoder trained for inversion by learning a posterior distribution of a fixed pre-trained GAN model, and the encoder using the fixed pre-trained GAN model as a decoder;

receiving an input image;

receiving an exemplar texture image;

generating, using the encoder, a first latent code vector representation based on the input image;

generating, using the first GAN generator, a second latent code vector representation based on the exemplar texture image;

blending the first latent code vector representation and the second latent code vector representation to obtain a blended latent code vector representation by concatenating a first predetermined amount of first sub-codes of the first latent code vector representation and a second predetermined amount of last sub-codes of the second latent code vector representation;

generating, by the first GAN generator, a texturized image based on the blended latent code vector representation; and

providing the texturized image as an output.

2. The method of claim 1 , wherein concatenating the first predetermined amount of first sub-codes of the first latent code vector representation and the second predetermined amount of last sub-codes of the second latent code vector representation comprises concatenating the first eight sub-codes of the first latent code vector representation and the last ten sub-codes of the second latent code vector representation.

3. The method of claim 1 , wherein the encoder is a hierarchical variational autoencoder.

4. The method of claim 3 , wherein the first latent code vector representation comprises 18×512 dimensions.

5. The method of claim 1 , wherein the first GAN generator is an AgileGAN generator.

6. The method of claim 1 , wherein training the encoder and training the first GAN generator are executed independently and in parallel.

7. A system, comprising:

one or more hardware processors configured by machine-readable instructions to:

receive a plurality of exemplar stylistic images;

train a first generative adversarial network (GAN) generator using transfer learning based on the received plurality of exemplar stylistic images;

receive a plurality of training images;

train an encoder using the plurality of training images and second GAN generator, the encoder trained for inversion by learning a posterior distribution of a fixed pre-trained GAN model, and the encoder using the fixed pre-trained GAN model as a decoder;

receive an input image;

receive an exemplar texture image;

generate, using the encoder, a first latent code vector representation based on the input image;

generate, using the first GAN generator, a second latent code vector representation based on the exemplar texture image;

blend the first latent code vector representation and the second latent code vector representation to obtain a blended latent code vector representation by concatenating a first predetermined amount of first sub-codes of the first latent code vector representation and a second predetermined amount of last sub-codes of the second latent code vector representation;

generate, by the first GAN generator, a texturized image based on the blended latent code vector representation; and

provide the texturized image as an output.

8. The system of claim 7 , wherein concatenating the first predetermined amount of first sub-codes of the first latent code vector representation and the second predetermined amount of last sub-codes of the second latent code vector representation comprises concatenating the first eight sub-codes of the first latent code vector representation and the last ten sub-codes of the second latent code vector representation.

9. The system of claim 7 , wherein the encoder is a hierarchical variational autoencoder.

10. The system of claim 9 , wherein the first latent code vector representation comprises 18×512 dimensions.

11. The system of claim 7 , wherein the first GAN generator is an AgileGAN generator.

12. The system of claim 7 , wherein training the encoder and training the first GAN generator are executed independently and in parallel.

13. A non-transitory computer-readable storage medium comprising instructions being executable by one or more processors to cause the one or more processors to:

receive a plurality of exemplar stylistic images;

train a first generative adversarial network (GAN) generator using transfer learning based on the received plurality of exemplar stylistic images;

receive a plurality of training images;

train an encoder using the plurality of training images and second GAN generator, the encoder trained for inversion by learning a posterior distribution of a fixed pre-trained GAN model, and the encoder using the fixed pre-trained GAN model as a decoder;

receive an input image;

receive an exemplar texture image;

generate, using the encoder, a first latent code vector representation based on the input image;

generate, using the first GAN generator, a second latent code vector representation based on the exemplar texture image;

blend the first latent code vector representation and the second latent code vector representation to obtain a blended latent code vector representation by concatenating a first predetermined amount of first sub-codes of the first latent code vector representation and a second predetermined amount of last sub-codes of the second latent code vector representation;

generate, by the first GAN generator, a texturized image based on the blended latent code vector representation; and

provide the texturized image as an output.

14. The computer-readable storage medium of claim 13 , wherein concatenating the first predetermined amount of first sub-codes of the first latent code vector representation and the second predetermined amount of last sub-codes of the second latent code vector representation comprises concatenating the first eight sub-codes of the first latent code vector representation and the last ten sub-codes of the second latent code vector representation.

15. The computer-readable storage medium of claim 13 , wherein the encoder is a hierarchical variational autoencoder.

16. The computer-readable storage medium of claim 15 , wherein the first latent code vector representation comprises 18×512 dimensions.

17. The computer-readable storage medium of claim 13 , wherein the first GAN generator is an AgileGAN generator.

18. The computer-readable storage medium of claim 13 , wherein training the encoder and training the first GAN generator are executed independently and in parallel.