Optimizing generative networks via latent space regularizations
A method for image generation based on a Generative AI Network. The Generative AI Network includes a generator and an encoder. The method includes determining, by the encoder, a first encoding E(Y) of a target image Y; generating, by the generator, a generated image G(Z) corresponding to the target image Y, wherein the generated image G(Z) is located in a close vicinity of a target neighborhood of the target image Y, and outputs of the generator are mapped, by the encoder, to a latent space adaptable to manipulate at least one characteristics of images generated by the Generative AI Network; and generating, by the encoder, a second encoding E(G(Z)) of the generated image G(Z) corresponding to the target image Y, wherein the first and second encodings E(Y) and E(G(Z)) map the target image Y and the generated image G(Z) to the latent space.
1. A method for image generation based on a Generative AI Network, the Generative AI Network comprising a generator and an encoder, the method comprising:
determining, by the encoder, a first encoding E(Y) of a target image Y;
generating, by the generator, a generated image G(Z) corresponding to the target image Y, wherein the generated image G(Z) is located in a close vicinity of a target neighborhood of the target image Y, and outputs of the generator are mapped, by the encoder, to a latent space adaptable to manipulate at least one characteristics of images generated by the Generative AI Network; and
generating, by the encoder, a second encoding E(G(Z)) of the generated image G(Z) corresponding to the target image Y, wherein the first and second encodings E(Y) and E(G(Z)) map the target image Y and the generated image G(Z) to the latent space.
2. The method of claim 1 , wherein the encoder is trained to minimize the differences between the first and second encodings E(Y) and E(G(Z)), and the generator is trained by using the first and second encodings E(Y) and E(G(Z)) as part of a loss function of the generator.
3. The method of claim 1 , wherein a same set of weights are used for first encoding E(Y) of the target image Y and the second encoding E(G(Z)) of the generated image G(Z) corresponding to the target image Y, and the same set of weights are updated after one or more iterations of training the Generative AI Network.
4. The method of claim 1 , wherein the first encoding E(Y) comprises a compressed representation of the target image Y in the latent space, and the second encoding E(G(Z)) comprises a compressed representation of the generated image G(Z) corresponding to the target image Y in the latent space.
5. The method of claim 4 , wherein the compressed representation of the target image Y in the latent space comprises the most relevant feature extracted from the target image Y, and the compressed representation of the generated image G(Z) corresponding to the target image Y in the latent space comprises the most relevant feature extracted from the generated image G(Z).
6. The method of claim 1 , wherein the encoder implements a machine learning model.
7. The method of claim 1 , wherein the encoder implements a convolutional neural network.
8. The method of claim 1 , wherein the encoder implements a VGG network.
9. The method of claim 1 , wherein the generator implements an inverse convolutional network.
10. The method of claim 1 , wherein a first loss function between the first encoding E(Y) and the second encoding E(G(Z)) is minimized for the encoder; and
the generator is trained using the first encoding E(Y) and the second encoding E(G(Z)) as part of a second loss function for the generator.
11. An apparatus comprising:
at least one processor; and
at least one memory, wherein the at least one memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to implement a Generative AI Network, the Generative AI Network comprising a generator and an encoder and to train the Generative AI Network by:
determining, by the encoder, a first encoding E(Y) of a target image Y;
generating, by the generator, a generated image G(Z) corresponding to the target image Y, wherein the generated image G(Z) is located in a close vicinity of a target neighborhood of the target image Y, and outputs of the generator are mapped, by the encoder, to a latent space adaptable to manipulate at least one characteristics of images generated by the Generative AI Network; and
generating, by the encoder, a second encoding E(G(Z)) of the generated image G(Z) corresponding to the target image Y, wherein the first and second encodings E(Y) and E(G(Z)) map the target image Y and the generated image G(Z) to the latent space.
12. The apparatus of claim 11 , the encoder is trained to minimize the differences between the first and second encodings E(Y) and E(G(Z)), and the generator is trained by using the first and second encodings E(Y) and E(G(Z)) as part of a loss function of the generator.
13. The apparatus of claim 11 , wherein a same set of weights are used for first encoding E(Y) of the target image Y and the second encoding E(G(Z)) of the generated image G(Z) corresponding to the target image Y, and the same set of weights are updated after one or more iterations of training the Generative AI Network.
14. The apparatus of claim 11 , wherein the first encoding E(Y) comprises a compressed representation of the target image Y in the latent space, and the second encoding E(G(Z)) comprises a compressed representation of the generated image G(Z) corresponding to the target image Y in the latent space.
15. The apparatus of claim 14 , wherein the compressed representation of the target image Y in the latent space comprises the most relevant feature extracted from the target image Y, and the compressed representation of the generated image G(Z) corresponding to the target image Y in the latent space comprises the most relevant feature extracted from the generated image G(Z).
16. The apparatus of claim 11 , wherein the encoder implements a machine learning model.
17. The apparatus of claim 11 , wherein the encoder implements a convolutional neural network.
18. The apparatus of claim 11 , wherein the encoder implements a VGG network.
19. The apparatus of claim 11 , wherein the generator implements an inverse convolutional network.
20. The apparatus of claim 11 , wherein a first loss function between the first encoding E(Y) and the second encoding E(G(Z)) is minimized for the encoder; and
the generator is trained using the first encoding E(Y) and the second encoding E(G(Z)) as part of a second loss function for the generator.