Variant inconsistency attack (VIA) as a simple and effective adversarial attack method
The present disclosure relates to techniques for using variant inconsistency attack (VIA) as a simple and effective adversarial attack method to create useful adversarial examples for adversarial training of machine-learning models. In one particular aspect, a method is provided that includes obtaining a set of input examples for attacking a machine-learning model (the set of examples do not have corresponding labels), modifying an example from the set of examples in a utility preserving manner to generate a pair of modified examples, attacking the machine-learning model with the pair of modified examples in order generate a pair of predictions for the pair of modified examples, comparing the pair of predictions to determine whether the pair of predictions are the same or different, and in response to the pair of predictions being different, adding the pair of modified examples to a set of adversarial examples.
1. A computer-implemented method comprising:
obtaining a set of input examples for attacking a machine-learning model, wherein the set of examples do not have corresponding labels;
modifying an example from the set of examples in a utility preserving manner to generate a pair of modified examples, wherein the utility preserving manner includes modifying the example without changing an original meaning or expected label of the example, and wherein the pair of modified examples share a same expected label;
attacking the machine-learning model with the pair of modified examples, wherein the attacking comprises:
inputting the pair of modified examples into the machine-learning model while maintaining the same expected label unknown to the machine-learning model, and
generating, by the machine-learning model, a pair of predictions for the pair of modified examples,
comparing the pair of predictions to determine whether the pair of predictions are the same or different;
in response to the pair of predictions being different, adding the pair of modified examples to a set of adversarial examples; and
training or fine-tuning the machine-learning model or a different machine-learning model using the set of adversarial examples, wherein the training or fine-tuning comprises annotating each example in the set of adversarial examples with an expected label to obtain annotated adversarial examples, feeding the annotated adversarial examples to the machine-learning model or the different machine-learning model, and learning or updating the plurality model parameters to account for characteristics of the set of adversarial examples based on maximizing or minimizing an objective function.
2. The method of claim 1 , further comprising in response to the pair of predictions being different and prior to adding the pair of modified examples to the set of adversarial example pairs, determining individually whether each modified example of the pair of modified examples is a true adversarial example, and limiting the addition of the modified examples to the set of adversarial examples as only being the addition of the modified examples deemed to be true adversarial examples to the set of adversarial examples.
3. The method of claim 1 , wherein the machine-learning model is part of a chatbot system.
4. The method of claim 1 , wherein the machine-learning model is a named entity recognition model, an intent recognition model, or an image recognition model.
5. The method of claim 1 , wherein the modifying comprises adding noise into the example.
6. The method of claim 1 , wherein the modifying comprises selecting one or more letters or words from the example, and performing an insert, delete, or replace operation on the one or more letters or words.
7. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, the memory storing a plurality of instructions executable by the one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to perform processing comprising:
obtaining a set of input examples for attacking a machine-learning model, wherein the set of examples do not have corresponding labels;
modifying an example from the set of examples in a utility preserving manner to generate a pair of modified examples, wherein the utility preserving manner includes modifying the example without changing an original meaning or expected label of the example, and wherein the pair of modified examples share a same expected label;
attacking the machine-learning model with the pair of modified examples, wherein the attacking comprises:
inputting the pair of modified examples into the machine-learning model while maintaining the same expected label unknown to the machine-learning model, and
generating, by the machine-learning model, a pair of predictions for the pair of modified examples,
comparing the pair of predictions to determine whether the pair of predictions are the same or different;
in response to the pair of predictions being different, adding the pair of modified examples to a set of adversarial examples; and
training or fine-tuning the machine-learning model or a different machine-learning model using the set of adversarial examples, wherein the training or fine-tuning comprises annotating each example in the set of adversarial examples with an expected label to obtain annotated adversarial examples, feeding the annotated adversarial examples to the machine-learning model or the different machine-learning model, and learning or updating the plurality model parameters to account for characteristics of the set of adversarial examples based on maximizing or minimizing an objective function.
8. The system of claim 7 , wherein the processing further comprises in response to the pair of predictions being different and prior to adding the pair of modified examples to the set of adversarial example pairs, determining individually whether each modified example of the pair of modified examples is a true adversarial example, and limiting the addition of the modified examples to the set of adversarial examples as only being the addition of the modified examples deemed to be true adversarial examples to the set of adversarial examples.
9. The system of claim 7 , wherein the machine-learning model is part of a chatbot system.
10. The system of claim 7 , wherein the machine-learning model is a named entity recognition model, an intent recognition model, or an image recognition model.
11. The system of claim 7 , wherein the modifying comprises adding noise into the example.
12. The system of claim 7 , wherein the modifying comprises selecting one or more letters or words from the example, and performing an insert, delete, or replace operation on the one or more letters or words.
13. A non-transitory computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to perform processing comprising:
obtaining a set of input examples for attacking a machine-learning model, wherein the set of examples do not have corresponding labels;
modifying an example from the set of examples in a utility preserving manner to generate a pair of modified examples, wherein the utility preserving manner includes modifying the example without changing an original meaning or expected label of the example, and wherein the pair of modified examples share a same expected label;
attacking the machine-learning model with the pair of modified examples, wherein the attacking comprises:
inputting the pair of modified examples into the machine-learning model while maintaining the same expected label unknown to the machine-learning model, and
generating, by the machine-learning model, a pair of predictions for the pair of modified examples,
comparing the pair of predictions to determine whether the pair of predictions are the same or different;
in response to the pair of predictions being different, adding the pair of modified examples to a set of adversarial examples; and
training or fine-tuning the machine-learning model or a different machine-learning model using the set of adversarial examples, wherein the training or fine-tuning comprises annotating each example in the set of adversarial examples with an expected label to obtain annotated adversarial examples, feeding the annotated adversarial examples to the machine-learning model or the different machine-learning model, and learning or updating the plurality model parameters to account for characteristics of the set of adversarial examples based on maximizing or minimizing an objective function.
14. The non-transitory computer-readable memory of claim 13 , wherein the processing further comprises in response to the pair of predictions being different and prior to adding the pair of modified examples to the set of adversarial example pairs, determining individually whether each modified example of the pair of modified examples is a true adversarial example, and limiting the addition of the modified examples to the set of adversarial examples as only being the addition of the modified examples deemed to be true adversarial examples to the set of adversarial examples.
15. The non-transitory computer-readable memory of claim 13 , wherein the machine-learning model is a named entity recognition model, an intent recognition model, or an image recognition model.
16. The non-transitory computer-readable memory of claim 13 , wherein the modifying comprises adding noise into the example.
17. The non-transitory computer-readable memory of claim 13 , wherein the modifying comprises selecting one or more letters or words from the example, and performing an insert, delete, or replace operation on the one or more letters or words.