Defense against adversarial example input to machine learning models
An apparatus comprises a processing device configured to train first and second machine learning models utilizing a first training dataset comprising inputs each associated with a class label of one of a set of classes and a second training dataset comprising distilled representations of the two or more classes, and to identify candidate adversarial example inputs utilizing the trained first and second machine learning models. The processing device is further configured to determine whether the candidate adversarial example inputs are true positive adversarial example inputs based on a confidence-aware clustering and to generate an updated first training dataset comprising corrected class labels for the true positive adversarial example inputs and an updated second training dataset comprising updated distilled representations determined utilizing the corrected class labels. The processing device is further configured to re-train the first and second machine learning models utilizing the updated first and second training datasets.
1 . An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured to perform steps of:
training a first machine learning model utilizing a first training dataset and training a second machine learning model utilizing a second training dataset, the first training dataset comprising a plurality of inputs each associated with a class label of one of a set of two or more classes, the second training dataset comprising distilled representations of each of at least a subset of the classes in the set of two or more classes, wherein the first machine learning model and the second machine learning model utilize a same neural network architecture, the neural network architecture comprising one or more hidden layers and an output layer;
identifying a plurality of candidate adversarial example inputs utilizing the trained first and second machine learning models, wherein identifying a given one of the plurality of candidate adversarial example inputs utilizing the trained first and second machine learning models is based at least in part on a similarity of a first output classification for the given candidate adversarial example input generated utilizing the trained first machine learning model and a second output classification for the given candidate adversarial example input generated utilizing the trained second machine learning model;
determining whether respective ones of the plurality of candidate adversarial example inputs are true positive adversarial example inputs based at least in part on a confidence-aware clustering of the plurality of candidate adversarial example inputs, the confidence-aware clustering being based at least in part on (i) generating, using processor-based adversarial example identification logic, soft label information for the plurality of candidate adversarial example inputs, (ii) generating two or more clusters of the plurality of candidate adversarial example inputs utilizing the soft label information generated for the plurality of candidate adversarial example inputs as features, and (iii) identifying true positive ones of the one or more candidate adversarial example inputs based at least in part on a weighted clustering algorithm that determines distances between ones of the plurality of candidate adversarial example inputs and centers of each of the generated two or more clusters of the plurality of candidate adversarial example inputs;
generating updated first and second training datasets, the updated first training dataset comprising corrected class labels for ones of the plurality of candidate adversarial example inputs determined to be true positive adversarial example inputs, the updated second training dataset comprising updated distilled representations determined utilizing the corrected class labels for the ones of the plurality of candidate adversarial example inputs determined to be true positive adversarial example inputs; and
re-training the first machine learning model utilizing the updated first training dataset and re-training the second machine learning model utilizing the updated second training dataset.
2 . The apparatus of claim 1 wherein the class labels in the first training dataset comprise hard label information, and wherein the distilled representations in the second training dataset comprise the soft label information.
3 . The apparatus of claim 1 wherein the distilled representations in the second training dataset comprise prototypes of feature information for each of at least the subset of the classes in the set of two or more classes.
4 . The apparatus of claim 1 wherein the first machine learning model comprises a capsule neural network, and wherein the distilled representations in the second training dataset are derived from patterns in the first training dataset learned by the trained capsule neural network.
5 . The apparatus of claim 1 wherein identifying the given candidate adversarial example input comprises determining whether a first output classification for the given candidate adversarial example input generated utilizing the trained first machine learning model is different than a second output classification for the given candidate adversarial example input generated utilizing the trained second machine learning model.
6 . The apparatus of claim 1 wherein the confidence-aware clustering of the plurality of candidate adversarial example inputs is based on confidence scores for the plurality of candidate adversarial example inputs, a given confidence score for the given candidate adversarial example input being determined based at least in part on a similarity metric characterizing the similarity of first and second output classifications for the given candidate adversarial example input generated by the trained first and second machine learning models.
7 . The apparatus of claim 6 wherein the similarity metric comprises a cosine similarity metric.
8 . The apparatus of claim 1 wherein the confidence-aware clustering utilizes a K-means clustering algorithm.
9 . The apparatus of claim 1 wherein the confidence-aware clustering comprises, for the given candidate adversarial example input, determining probabilities that the given candidate adversarial example input belongs to each class in the set of two or more classes.
10 . The apparatus of claim 1 wherein re-training the first and second machine learning models is based at least in part on confidence scores for the ones of the plurality of candidate adversarial example inputs determined to be true positive adversarial example inputs.
11 . The apparatus of claim 10 wherein the re-training of the first and second machine learning models has a loss determined based at least in part on ratios of the confidence scores for the ones of the plurality of candidate adversarial example inputs determined to be true positive adversarial example inputs and an original classification loss during the training of the first and second machine learning models.
12 . The apparatus of claim 1 wherein the at least one processing device is further configured to utilize at least one of the re-trained first and second machine learning models for performing classification of subsequent input.
13 . A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform steps of:
training a first machine learning model utilizing a first training dataset and training a second machine learning model utilizing a second training dataset, the first training dataset comprising a plurality of inputs each associated with a class label of one of a set of two or more classes, the second training dataset comprising distilled representations of each of at least a subset of the classes in the set of two or more classes, wherein the first machine learning model and the second machine learning model utilize a same neural network architecture, the neural network architecture comprising one or more hidden layers and an output layer;
identifying a plurality of candidate adversarial example inputs utilizing the trained first and second machine learning models, wherein identifying a given one of the plurality of candidate adversarial example inputs utilizing the trained first and second machine learning models is based at least in part on a similarity of a first output classification for the given candidate adversarial example input generated utilizing the trained first machine learning model and a second output classification for the given candidate adversarial example input generated utilizing the trained second machine learning model;
determining whether respective ones of the plurality of candidate adversarial example inputs are true positive adversarial example inputs based at least in part on a confidence-aware clustering of the plurality of candidate adversarial example inputs, the confidence-aware clustering being based at least in part on (i) generating, using processor-based adversarial example identification logic, soft label information for the plurality of candidate adversarial example inputs, (ii) generating two or more clusters of the plurality of candidate adversarial example inputs utilizing the soft label information generated for the plurality of candidate adversarial example inputs as features, and (iii) identifying true positive ones of the one or more candidate adversarial example inputs based at least in part on a weighted clustering algorithm that determines distances between ones of the plurality of candidate adversarial example inputs and centers of each of the generated two or more clusters of the plurality of candidate adversarial example inputs;
generating updated first and second training datasets, the updated first training dataset comprising corrected class labels for ones of the plurality of candidate adversarial example inputs determined to be true positive adversarial example inputs, the updated second training dataset comprising updated distilled representations determined utilizing the corrected class labels for the ones of the plurality of candidate adversarial example inputs determined to be true positive adversarial example inputs; and
re-training the first machine learning model utilizing the updated first training dataset and re-training the second machine learning model utilizing the updated second training dataset.
14 . The computer program product of claim 13 wherein identifying the given candidate adversarial example input comprises determining whether a first output classification for the given candidate adversarial example input generated utilizing the trained first machine learning model is different than a second output classification for the given candidate adversarial example input generated utilizing the trained second machine learning model.
15 . The computer program product of claim 13 wherein the confidence-aware clustering of the plurality of candidate adversarial example inputs is based on confidence scores for the plurality of candidate adversarial example inputs, a given confidence score for the given candidate adversarial example input being determined based at least in part on a similarity metric characterizing the similarity of first and second output classifications for the given candidate adversarial example input generated by the trained first and second machine learning models.
16 . A method comprising:
training a first machine learning model utilizing a first training dataset and training a second machine learning model utilizing a second training dataset, the first training dataset comprising a plurality of inputs each associated with a class label of one of a set of two or more classes, the second training dataset comprising distilled representations of each of at least a subset of the classes in the set of two or more classes, wherein the first machine learning model and the second machine learning model utilize a same neural network architecture, the neural network architecture comprising one or more hidden layers and an output layer;
identifying a plurality of candidate adversarial example inputs utilizing the trained first and second machine learning models, wherein identifying a given one of the plurality of candidate adversarial example inputs utilizing the trained first and second machine learning models is based at least in part on a similarity of a first output classification for the given candidate adversarial example input generated utilizing the trained first machine learning model and a second output classification for the given candidate adversarial example input generated utilizing the trained second machine learning model;
determining whether respective ones of the plurality of candidate adversarial example inputs are true positive adversarial example inputs based at least in part on a confidence-aware clustering of the plurality of candidate adversarial example inputs, the confidence-aware clustering being based at least in part on (i) generating, using processor-based adversarial example identification logic, soft label information for the plurality of candidate adversarial example inputs, (ii) generating two or more clusters of the plurality of candidate adversarial example inputs utilizing the soft label information generated for the plurality of candidate adversarial example inputs as features, and (iii) identifying true positive ones of the one or more candidate adversarial example inputs based at least in part on a weighted clustering algorithm that determines distances between ones of the plurality of candidate adversarial example inputs and centers of each of the generated two or more clusters of the plurality of candidate adversarial example inputs;
generating updated first and second training datasets, the updated first training dataset comprising corrected class labels for ones of the plurality of candidate adversarial example inputs determined to be true positive adversarial example inputs, the updated second training dataset comprising updated distilled representations determined utilizing the corrected class labels for the ones of the plurality of candidate adversarial example inputs determined to be true positive adversarial example inputs; and
re-training the first machine learning model utilizing the updated first training dataset and re-training the second machine learning model utilizing the updated second training dataset;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.
17 . The method of claim 16 wherein identifying the given candidate adversarial example input comprises determining whether a first output classification for the given candidate adversarial example input generated utilizing the trained first machine learning model is different than a second output classification for the given candidate adversarial example input generated utilizing the trained second machine learning model.
18 . The method of claim 16 wherein the confidence-aware clustering of the one or more candidate adversarial example inputs is based on confidence scores for the plurality of candidate adversarial example inputs, a given confidence score for a given one of the plurality of candidate adversarial example inputs being determined based at least in part on a similarity metric characterizing similarity of first and second output classifications for the given candidate adversarial example input generated by the trained first and second machine learning models.
19 . The method of claim 16 wherein the class labels in the first training dataset comprise hard label information, and wherein the distilled representations in the second training dataset comprise the soft label information.
20 . The method of claim 16 wherein the distilled representations in the second training dataset comprise prototypes of feature information for each of at least the subset of the classes in the set of two or more classes.