Latent code for unsupervised domain adaptation

1. A computer-implemented method of training a target encoder for classifying images into one of a plurality of categories using a classifier trained on images from a source domain, the method comprising:

generating, by a pre-trained source encoder, a source encoder representation for each image of a labeled set of source images from the source domain;

generating by the target encoder, a target encoder representation for each image of an unlabeled set of target images from a target domain;

inputting, into a generative adversarial network, the source encoder representations, the target encoder representations, and latent code representing the plurality of categories;

by the generative adversarial network, outputting a first prediction indicating whether each of the source encoder representations and each of the target encoder representations originate from the source domain or the target domain;

by the generative adversarial network, outputting a second prediction of the latent code for each of the source encoder representations and each of the target encoder representations; and

training the target encoder and the generative adversarial network by repeatedly updating parameters of the target encoder and the generative adversarial network until one or more predetermined stopping conditions occur.

2. The method of claim 1 , wherein the training includes training the target encoder and the generative adversarial network by repeatedly updating parameters of the target encoder and the generative adversarial network until a first loss function for the first prediction reaches a minimum value and a second loss function for the second prediction reaches a minimum value.

3. The method claim 1 , further comprising:

classifying a target image using the trained target encoder and the classifier, wherein the classifier is trained using supervised learning on the labeled set of source images.

4. The method of claim 3 , wherein classifying a target image using the trained target encoder and the classifier, comprises:

inputting a target image into the trained target encoder;

generating a target encoder representation of the target image;

inputting the target encoder representation of the target image into the trained classifier; and

classifying the target image by the trained classifier into one of the plurality of categories.

5. The method of claim 1 , further comprising pre-training the source encoder and the classifier by:

inputting the source encoder representations and source labels into the classifier; and

pre-training the source encoder and the classifier by repeatedly updating parameters of the source encoder and the classifier until one or more second predetermined stopping conditions occur.

6. The method of claim 5 , wherein the pre-training includes pre-training the source encoder and the classifier by repeatedly updating parameters of the source encoder and the classifier until a classification loss function reaches a minimum value.

7. The method of claim 5 , wherein the parameters of the pre-trained source encoder are not fixed after the pre-training, and

wherein the method further comprises training the pre-trained source encoder jointly with the target encoder and the generative adversarial network by repeatedly updating parameters of the source encoder, the target encoder, and the generative adversarial network (A) until a classification loss function reaches a minimum value, a first loss function for the first prediction reaches a minimum value and a second loss function for the second prediction reaches a minimum value.

8. The method of claim 5 , wherein the parameters of the pre-trained source encoder are fixed after the pre-training.

9. The method of claim 1 , wherein the parameters of the pre-trained source encoder are fixed after the pre-training.

10. The method of claim 1 , wherein the parameters of the target encoder are initialized using parameters of the pre-trained source encoder.

11. The method of claim 1 , wherein the generative adversarial network comprises a generator neural network, a discriminator neural network, and an auxiliary neural network, and

wherein the inputting, into the generative adversarial network, the source encoder representations, the target encoder representations, and the latent code comprises inputting, into the generator neural network, the source encoder representations combined with the latent code and the target encoder representations combined with the latent code.

12. The method of claim 11 , further comprising:

computing, by the generator neural network, joint hidden representations for each of the source encoder representations and each of the target encoder representations, each joint hidden representation based on a combination of the latent code and a respective one of the source encoder representations and the target encoder representations;

inputting, into the discriminator neural network, the joint hidden representations, wherein the discriminator neural network is configured to output the first prediction; and

inputting, into the auxiliary neural network, the joint hidden representations, wherein the auxiliary neural network is configured to output the second prediction.

13. The method of claim 12 wherein outputting, by the auxiliary neural network, the second prediction of the latent code for each of the joint hidden representations comprises outputting a probability distribution for the latent code given the respective joint representation.

14. The method of claim 13 , wherein the auxiliary neural network includes a convolutional neural network comprising a final fully connected layer and a softmax function to output parameters for the probability distribution.

15. The method of claim 11 , wherein the generator neural network includes a convolutional neural network comprising three hidden layers with rectified linear unit (ReLu) activation.

16. The method of claim 11 , wherein the discriminator neural network includes a convolutional neural network comprising at least one fully connected layer.

17. A non-transitory computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by one or more processors perform a method of training a target encoder for classifying images into one of a plurality of categories using a classifier trained on images from a source domain, the method comprising:

generating, by a pre-trained source encoder, a source encoder representation for each image of a labeled set of source images from the source domain;

generating, by the target encoder, a target encoder representation for each image of an unlabeled set of target images from a target domain;

inputting, into a generative adversarial network, the source encoder representations, the target encoder representations, and latent code representing the plurality of categories,

wherein the generative adversarial network is configured to:

output a first prediction indicating whether each of the source encoder representations and each of the target encoder representations originate from the source domain or the target domain; and

output a second prediction of the latent code for each of the source encoder representations and each of the target encoder representations; and

training the target encoder and the generative adversarial network by repeatedly updating parameters of the target encoder and the generative adversarial network until one or more predetermined stopping conditions occur.

18. A training system comprising:

a target encoder configured to classify images into one of a plurality of categories using a classifier;

a pre-trained source encoder configured to generate a source encoder representation for each image of a labeled set of source images from a source domain,

wherein the target encoder is configured to generate a target encoder representation for each image of an unlabeled set of target images from a target domain;

a generative adversarial network configured to:

receive the source encoder representations, the target encoder representations, and latent code representing the plurality of categories;

output a first prediction indicating whether each of the source encoder representations and each of the target encoder representations originate from the source domain or the target domain;

output a second prediction of the latent code for each of the source encoder representations and each of the target encoder representations; and

one or more processors configured to train the target encoder and the generative adversarial network by repeatedly updating parameters of the target encoder and the generative adversarial network until one or more predetermined stopping conditions occur.

19. The training system of claim 18 wherein the one or more processors are configured to train the target encoder and the generative adversarial network by repeatedly updating parameters of the target encoder and the generative adversarial network until a first loss function for the first prediction reaches a minimum value and a second loss function for the second prediction reaches a minimum value.

20. The training system of claim 18 wherein the generative adversarial network comprises a generator neural network, a discriminator neural network, and an auxiliary neural network.

21. The training system of claim 20 wherein:

the generator neural network is configured to generate joint hidden representations for each of the source encoder representations and each of the target encoder representations, each joint hidden representation generated based on a combination of the latent code and a respective one of the source encoder representations and the target encoder representations;

the discriminator neural network is configured to generate the first prediction based on the joint hidden representations; and

the auxiliary neural network is configured to output the second prediction based on the joint hidden representations.

22. The training system of claim 21 , wherein the auxiliary neural network is configured to output the second prediction including a probability distribution for the latent code given the respective joint representation.

23. The training system of claim 22 , wherein:

the auxiliary neural network includes a convolutional neural network comprising a final fully connected layer and a softmax function to output parameters for the probability distribution;

the generator neural network includes a convolutional neural network comprising three hidden layers with rectified linear unit (ReLu) activation; and

the discriminator neural network includes a convolutional neural network comprising at least one fully connected layer.