Clustering autoencoder
Discussed herein are devices, systems, and methods for classification using a clustering autoencoder. A method can include receiving, by an encoder of an autoencoder, content, the autoencoder trained using other content and corresponding labels, providing, by the encoder, a latent feature representation of the content to a decoder of the autoencoder, providing, by a clustering layer situated between the encoder and the decoder, a probability that the content belongs to a class of classes represented by respective clusters in a latent feature representation space based on a distance between the feature representation and the cluster, and providing, by the decoder, reconstructed content that is a construction of the content based on the latent feature representation.
1. A computer-implemented method for content classification using supervised machine learning (ML), the method comprising:
receiving, by an encoder of an autoencoder, content, the autoencoder trained using other content and corresponding labels, the autoencoder including an encoder, decoder, and a clustering layer, the clustering layer situated between the encoder and the decoder;
providing, by the encoder, a latent feature representation in a latent feature space of the content to the decoder and the clustering layer;
determining, by the clustering layer, respective confidences based on respective distances between the latent feature representation and respective representative points in respective clusters, the respective confidences indicating respective probabilities that the content belongs to the respective clusters, each of the respective clusters representing a different respective class of respective classes in the latent feature space;
providing, by the clustering layer, the respective confidences and corresponding classes of the respective classes; and
providing, by the decoder and based on the latent feature representation, reconstructed content that is a construction of the content.
2. The computer-implemented method of claim 1 , wherein the autoencoder is trained to reduce a difference between the content and the reconstructed content and increase a similarity between the labels and predicted class probabilities.
3. The computer-implemented method of claim 2 , wherein the autoencoder is trained based on an L2 norm and a cross-entropy.
4. The computer-implemented method of claim 1 , wherein the probability that the content belongs to the class is determined using a Student's t-Distribution or a mixture of Gaussian distributions.
5. The computer-implemented method of claim 1 , further comprising sampling a latent feature representation from a cluster of the clusters to determine another member of the class.
6. The computer-implemented method of claim 5 , further comprising:
wherein the probability that the content belongs to the class is determined using a Student's t-Distribution;
fitting a mixture of Gaussian distributions to the Student's t-Distribution; and
sampling a latent feature representation from a cluster of the clusters using the mixture of Gaussian distributions to determine another member of the class.
7. The computer-implemented method of claim 1 , further comprising:
determining a reconstruction error based on the reconstructed and the content; and
determining the content is not a member of the class in response to determining the reconstruction error is greater than a threshold.
8. The computer-implemented method of claim 7 , wherein the content is computer-transmissible content and the classes include respective malicious classes.
9. The computer-implemented method of claim 8 , further comprising determining a normalized probability complement plus the reconstruction error to determine the probability that the latent feature representation is a member of the class.
10. The computer-implemented method of claim 1 , further comprising determining the content is an outlier in response to the distance between the latent feature representation of the content being more than a threshold distance away from a central point of each of the clusters.
11. A non-transitory machine-readable medium including instructions that, when executed by a machine, cause the machine to perform operations for content classification using machine learning (ML), the operations comprising:
receiving, by an encoder of an autoencoder, content, the autoencoder trained using other content and corresponding labels, the autoencoder including an encoder, decoder, and a clustering layer, the clustering layer situated between the encoder and the decoder;
providing, by the encoder, a latent feature representation in a latent feature space of the content to the decoder and the clustering layer;
determining, by the clustering layer, respective confidences based on respective distances between the latent feature representation and respective representative points in respective clusters, the respective confidences indicating respective probabilities that the content belongs to the respective clusters, each of the respective clusters representing a different respective class of respective classes in the latent feature space;
providing, by the clustering layer, the respective confidences and corresponding classes of the respective classes; and
providing, by the decoder and based on the latent feature representation, reconstructed content that is a construction of the content.
12. The non-transitory machine-readable medium of claim 11 , wherein the autoencoder is trained to reduce a difference between the content and the reconstructed content and increase a similarity between the labels and predicted class probabilities.
13. The non-transitory machine-readable medium of claim 12 , wherein the autoencoder is trained based on an L2 norm and a cross-entropy.
14. The non-transitory machine-readable medium of claim 11 , wherein the probability that the content belongs to the class is determined using a Student's t-Distribution or a mixture of Gaussian distributions.
15. The non-transitory machine-readable medium of claim 11 , wherein the operations further comprise sampling a latent feature representation from a cluster of the clusters to determine another member of the class.
16. An autoencoder clustering system comprising:
a memory including instructions stored thereon;
processing circuitry configured to execute the instructions, the instruction, when executed by the processing circuitry cause the processing circuitry to implement the clustering autoencoder that:
receives, by an encoder of the autoencoder, content, the autoencoder trained using other content and corresponding labels, the autoencoder including an encoder, decoder, and a clustering layer situated between the encoder and the decoder, the autoencoder including an encoder, decoder, and a clustering layer, the clustering layer situated between the encoder and the decoder;
provides, by the encoder, a latent feature representation in a latent feature space of the content to the decoder;
determines, by the clustering layer, respective confidences based on respective distances between the latent feature representation and respective representative points in respective clusters, the respective confidences indicating respective probabilities that the content belongs to the respective clusters, each of the respective clusters representing a different respective class of respective classes in the latent feature space;
provides, by the clustering layer, the respective confidences and corresponding classes of the respective classes; and
provides, by the decoder and based on the latent feature representation, reconstructed content that is a construction of the content.
17. The system of claim 16 , wherein the instructions, when executed by the processing circuitry, further cause the processing circuitry to:
determine a reconstruction error based on the reconstructed and the content; and
determine the content is not a member of the class in response to determining the reconstruction error is greater than a threshold.
18. The system of claim 17 , wherein the content is computer-transmissible content and the classes include respective malicious classes.
19. The system of claim 18 , wherein the instructions, when executed by the processing circuitry, further cause the processing circuitry to determine a normalized probability complement plus the reconstruction error to determine the probability that the latent feature representation is a member of the class.
20. The of claim 16 , wherein the instructions, when executed by the processing circuitry, further cause the processing circuitry to determine the content is an outlier in response to the distance between the latent feature representation of the content being more than a threshold distance away from a central point of each of the clusters.