Transform disentangling auto-encoder and related methods
Discussed herein are devices, systems, and methods for disentangling static and dynamic features of content. A method can include encoding by a transform disentangling autoencoder (AE), first content to generate first static features and first dynamic features and second content to generate second static features and second dynamic features, and constructing, by the AE, third content based on a combination of third static features and the first dynamic features and fourth content based on a combination of fourth static features and the second dynamic features, the third and fourth static features being determined based on the first static features and the second static features.
1. A computer-implemented method for disentangling static features and dynamic features of content using an autoencoder (AE), the method comprising:
encoding, by the AE, first content to generate first static features and first dynamic features and second content to generate second static features and second dynamic features; and
constructing, by the AE, third content based on a combination of third static features and the first dynamic features and fourth content based on a combination of fourth static features and the second dynamic features, the third and fourth static features being determined based on the first static features and the second static features, the third static features different from the fourth static features.
2. The method of claim 1 , further comprising swapping, by the AE, one or more static features of the second static features with one or more static features of the first static features to generate the third static features and the fourth static features.
3. The method of claim 1 , further comprising training the AE using two or more images of an object, one of the two or more images including a transformed version of the object relative to at least one other image of the other two or more images.
4. The method of claim 3 , wherein training the AE includes reducing a loss function that accounts for generation loss between the first content and the third content, generation loss between the second content and the fourth content.
5. The method of claim 4 , wherein the loss function further includes a difference between the first static features and the second static features.
6. The method of claim 1 , further comprising generating, using a generative adversarial network (GAN), fifth content based on a combination of the first and second static features and the first and second dynamic features, and classifying, by the GAN, whether the third, fourth, and fifth content are real or fake.
7. The method of claim 6 , wherein generating the fifth content includes shuffling individual static features of the first and second static features.
8. The method of claim 6 , further comprising training the AE based on the classification.
9. The method of claim 1 , wherein the AE is fully convolutional.
10. The method of claim 1 , further comprising determining respective hash values of the first static features and the second static features and storing the hash values in a memory.
11. The method of claim 10 , further comprising:
encoding, by the AE, fifth content to generate third static features;
determining a second hash value of the third static features; and
comparing the second hash value to hash values of the first and second static features to determine whether the fifth content is like the first content or the second content.
12. A non-transitory machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations for disentangling static features and dynamic features of content, the operations comprising:
encoding, by an autoencoder (AE), first content to generate first static features and first dynamic features and second content to generate second static features and second dynamic features; and
constructing, by the AE, third content based on a combination of third static features and the first dynamic features and fourth content based on a combination of fourth static features and the second dynamic features, the third and fourth static features being determined based on the first static features and the second static features, the third static features different from the fourth static features.
13. The non-transitory machine-readable medium of claim 12 , wherein the operations further comprise swapping, by the AE, one or more static features of the second static features with one or more static features of the first static features to generate the third static features and the fourth static features.
14. The non-transitory machine-readable medium of claim 12 , wherein the operations further include training the AE using two or more images of an object, one of the two or more images including a transformed version of the object relative to at least one other image of the other two or more images.
15. The non-transitory machine-readable medium of claim 14 , wherein training the AE includes reducing a loss function that accounts for generation loss between the first content and the third content, generation loss between the second content and the fourth content.
16. The non-transitory machine-readable medium of claim 15 , wherein the loss function further includes a difference between the first static features and the second static features.
17. A system for disentangling static features and dynamic features of content, the system comprising:
a memory including instructions stored thereon;
processing circuitry configured to execute the instructions, the instruction, when executed by the processing circuitry cause the processing circuitry to implement a transform disentangling autoencoder (AE) that:
encodes first content to generate first static features and first dynamic features and second content to generate second static features and second dynamic features; and
constructs third content based on a combination of third static features and the first dynamic features and fourth content based on a combination of fourth static features and the second dynamic features, the third and fourth static features being determined based on the first static features and the second static features, the third static features different from the fourth static features.
18. The system of claim 17 , wherein the instructions include further instructions that cause the processing circuitry to implement a generative adversarial network (GAN) that generates fifth content based on a combination of the first and second static features and the first and second dynamic features, and classify whether the third, fourth, and fifth content are real or fake.
19. The system of claim 18 , wherein generating the fifth content includes shuffling individual static features of the first and second static features.
20. The system of claim 18 , wherein the instructions include further instructions that cause the processing circuitry train the AE and GAN based on the classification.