Using multimodal model consistency to detect adversarial attacks
A method, apparatus and computer program product to defend learning models that are vulnerable to adversarial example attack. It is assumed that data (a “dataset”) is available in multiple modalities (e.g., text and images, audio and images in video, etc.). The defense approach herein is premised on the recognition that the correlations between the different modalities for the same entity can be exploited to defend against such attacks, as it is not realistic for an adversary to attack multiple modalities. To this end, according to this technique, adversarial samples are identified and rejected if the features from one (the attacked) modality are determined to be sufficiently far away from those of another un-attacked modality for the same entity. In other words, the approach herein leverages the consistency between multiple modalities in the data to defend against adversarial attacks on one modality.
1. A method to protect a machine learning model from misclassification by adversarial attack, wherein the machine learning model is associated with a dataset having some duplicate information specified in at least first and second modalities that differ from one another, wherein a modality is a particular mode in which the duplicate information is expressed, comprising:
obtaining a first model and a second model, wherein the first model is specific to the first modality and the second model is specific to the second modality;
training a consistency checking model based on the first and second models;
receiving an input sample associated with data of the first and second modalities; and
rejecting the input sample when a consistency level between the first and second modalities has a given relationship to a threshold associated with the consistency checking model.
2. The method as described in claim 1 wherein training the consistency checking model comprises:
extracting a feature set from each of the first and second models;
projecting the feature set extracted from each model into a common feature space; and
using differences between the features sets to train the consistency checking model.
3. The method as described in claim 1 wherein the input sample is rejected when the consistency level between the first and second modalities is greater than the threshold.
4. The method as described in claim 2 wherein the feature set of each model is derived from outputs of a given fully-connected layer before logits layer for the respective model.
5. The method as described in claim 2 wherein the common feature space is associated with a projection model, wherein the projection model has an objective function that constrains a feature consistency check.
6. The method as described in claim 1 wherein, as compared to the second modality, the first modality has a higher risk of compromise by adversarial attack.
7. The method as described in claim 1 further including performing a classification using the machine learning model without the input sample.
8. An apparatus, comprising:
a processor;
computer memory holding computer program instructions executed by the processor to protect a machine learning model from misclassification by adversarial attack, the machine learning model being associated with a dataset having some duplicate information specified in at least first and second modalities that differ from one another, wherein a modality is a particular mode in which the duplicate information is expressed, the computer program instructions configured to:
obtain a first model and a second model, wherein the first model is specific to the first modality and the second model is specific to the second modality;
train a consistency checking model based on the first and second models;
receive an input sample associated with data of the first and second modalities; and
reject the input sample when a consistency level between the first and second modalities has a given relationship to a threshold associated with the consistency checking model.
9. The apparatus as described in claim 8 wherein the computer program instructions configured to train the consistency checking model comprise computer program instructions further configured to:
extract a feature set from each of the first and second models;
project the feature set extracted from each model into a common feature space; and
use differences between the features sets to train the consistency checking model.
10. The apparatus as described in claim 8 wherein the input sample is rejected when the consistency level between the first and second modalities is greater than a threshold.
11. The apparatus as described in claim 9 wherein the feature set of each model is derived from outputs of a given fully-connected layer before logits layer for the respective model.
12. The apparatus as described in claim 9 wherein the common feature space is associated with a projection model, wherein the projection model has an objective function that constrains a feature consistency check.
13. The apparatus as described in claim 8 wherein, as compared to the second modality, the first modality has a higher risk of compromise by adversarial attack.
14. The apparatus as described in claim 13 wherein the computer program instructions are further configured to perform erforming a classification using the machine learning model without the input sample.
15. A computer program product in a non-transitory computer readable medium for use in a data processing system to protect a machine learning model from misclassification by adversarial attack, the machine learning model being associated with a dataset having some duplicate information specified in at least first and second modalities that differ from one another, wherein a modality is a particular mode in which the duplicate information is expressed, the computer program product holding computer program instructions that, when executed by the data processing system, are configured to:
obtain a first model and a second model, wherein the first model is specific to the first modality and the second model is specific to the second modality;
train a consistency checking model based on the first and second models;
receive an input sample associated with data of the first and second modalities; and
reject the input sample when a consistency level between the first and second modalities has a given relationship to a threshold associated with the consistency checking model.
16. The computer program product as described in claim 15 wherein the computer program instructions configured to train the consistency checking model comprise computer program instructions further configured to:
extract a feature set from each of the first and second models;
project the feature set extracted from each model into a common feature space; and
use differences between the features sets to train the consistency checking model.
17. The computer program product as described in claim 16 wherein the input sample is rejected when the consistency level between the first and second modalities is greater than a threshold.
18. The computer program product as described in claim 16 wherein the feature set of each model is derived from outputs of a given fully-connected layer before logits layer for the respective model.
19. The computer program product as described in claim 16 wherein the common feature space is associated with a projection model, wherein the projection model has an objective function that constrains a feature consistency check.
20. The computer program product as described in claim 15 wherein, as compared to the second modality, the first modality has a higher risk of compromise by adversarial attack.
21. The computer program product as described in claim 20 wherein the computer program instructions are further configured to perform a classification using the machine learning model without the input sample.