Unsupervised representation learning with contrastive prototypes
The system and method are directed to a prototypical contrastive learning (PCL). The PCL explicitly encodes the hierarchical semantic structure of the dataset into the learned embedding space and prevents the network from exploiting low-level cues for solving the unsupervised learning task. The PCL includes prototypes as the latent variables to help find the maximum-likelihood estimation of the network parameters in an expectation-maximization framework. The PCL iteratively performs an E-step for finding prototypes with clustering and M-step for optimizing the network on a contrastive loss.
1. A method for training a prototypical contrastive learning (PCL) framework to cluster images, comprising:
receiving images at a momentum encoder of the PCL framework;
determining, using the momentum encoder, features from the images;
clustering, using the momentum encoder, the images into clusters according to the features;
determining at least one prototype and at least one concentration of the clusters;
determining, a contrastive loss function from the at least one prototype and the at least one concentration of the clusters;
training an encoder of the PCL framework using the contrastive loss function and a subset of images in the images, wherein the training updates the weights of the encoder based on the contrastive loss function;
updating weights of the momentum encoder using weights of the encoder; and
determining second clusters from second images using the updated momentum encoder.
2. The method of claim 1 , wherein the encoder is a convolutional neural network.
3. The method of claim 1 , wherein a neural network structure of the momentum encoder is the same as a neural network structure of the encoder.
4. The method of claim 1 , wherein the updated weights of the momentum encoder are a moving average of the weights of the momentum encoder and the weights of the encoder.
5. The method of claim 1 , further comprising:
dividing the images into multiple subsets of images; and
training the encoder of the PCL framework on the subsets of images until the contrastive loss function is minimized.
6. The method of claim 1 , wherein the updated momentum encoder is trained to perform at least one task associated with processing the images.
7. The method of claim 1 , further comprising:
training the encoder and updating the momentum encoder over a configurable number of iterations.
8. A system for training a prototypical contrastive learning (PCL) framework to cluster images, comprising:
at least one memory configured to store the PCL framework and images; and
a processor coupled to the memory and configured to execute instructions for training the PCL framework, the instructions comprising:
receiving the images at a momentum encoder of the PCL framework;
determining, using the momentum encoder, features from the images;
clustering, using the momentum encoder, the images into clusters according to the features;
determining at least one prototype and at least one concentration of the clusters;
determining, a contrastive loss function from the at least one prototype and the at least one concentration of the clusters;
training an encoder of the PCL framework using the contrastive loss function and a subset of images in the images;
updating weights of the momentum encoder using weights of the encoder; and
determining second clusters from second images using the updated momentum encoder.
9. The system of claim 8 , wherein the encoder is a convolutional neural network.
10. The system of claim 8 , wherein a neural network structure of the momentum encoder is the same as a neural network structure of the encoder.
11. The system of claim 8 , wherein the training updates the weights of the encoder based on the contrastive loss function.
12. The system of claim 8 , wherein the updated weights of the momentum encoder are a moving average of the weights of the momentum encoder and the weights of the encoder.
13. The system of claim 8 , wherein the processor is further configured to perform instructions comprising:
dividing the images into multiple subsets of images; and
training the encoder of the PCL framework on the subsets of images until the contrastive loss function is minimized.
14. The system of claim 8 , wherein the updated momentum encoder is trained to perform at least one task associated with processing the images.
15. The system of claim 8 , wherein the processor is further configured to perform instructions comprising:
training the encoder and updating the momentum encoder over a configurable number of iterations.
16. A non-transitory computer-readable medium storing instructions thereon, that when executed by a processor, cause the processor to perform operations that train a prototypical contrastive learning (PCL) framework to cluster images, the operations comprising:
receiving images at a momentum encoder of the PCL framework;
determining, using the momentum encoder, features from the images;
clustering, using the momentum encoder, the images into clusters according to the features;
determining at least one prototype and at least one concentration of the clusters;
determining, a contrastive loss function from the at least one prototype and the at least one concentration of the clusters;
training an encoder of the PCL framework using the contrastive loss function and a subset of images in the images, wherein the training updates the weights of the encoder based on the contrastive loss function;
updating weights of the momentum encoder using weights of the encoder; and
determining second clusters from second images using the updated momentum encoder.
17. The non-transitory computer-readable medium of claim 16 , wherein a neural network structure of the momentum encoder is the same as a neural network structure of the encoder.
18. The non-transitory computer-readable medium of claim 16 , wherein the updated weights of the momentum encoder are a moving average of the weights of the momentum encoder and the weights of the encoder.
19. The non-transitory computer-readable medium of claim 16 , wherein the operations further comprise:
dividing the images into multiple subsets of images; and
training the encoder of the PCL framework on the subsets of images until the contrastive loss function is minimized.
20. The non-transitory computer-readable medium of claim 16 , wherein the updated momentum encoder is trained to perform at least one task associated with processing the images.