Variant inconsistency attack (VIA) as a simple and effective adversarial attack method
The present disclosure relates to techniques for using variant inconsistency attack (VIA) as a simple and effective adversarial attack method to create useful adversarial examples for adversarial training of machine-learning models. In one particular aspect, a method is provided that includes obtaining a set of input examples for attacking a machine-learning model (the set of examples do not have corresponding labels), modifying an example from the set of examples in a utility preserving manner to generate a pair of modified examples, attacking the machine-learning model with the pair of modified examples in order generate a pair of predictions for the pair of modified examples, comparing the pair of predictions to determine whether the pair of predictions are the same or different, and in response to the pair of predictions being different, adding the pair of modified examples to a set of adversarial examples.
1 . A computer implemented method comprising:
receiving, at a digital assistant comprising a chatbot, one or more utterances from a user, wherein the one or more utterances comprise at least one utterance having noise;
inputting the one or more utterances into the chatbot;
processing, by the chatbot, the one or more utterances, wherein the processing comprises:
performing, by a machine learning model, one or more natural language tasks on the one or more utterances to obtain output, wherein the machine learning model is fine-tuned on adversarial examples in order to perform the one or more natural language tasks on utterances that have noise, wherein:
the adversarial examples comprise pairs of modified examples,
one of the examples in each pair of modified examples has been modified from the other of the examples in each respective pair of modified examples in a utility preserving manner,
the utility preserving manner includes modifying the example without changing an original meaning or expected label of the example, and
the pair of modified examples share a same expected label, and
generating, based on the output, a natural language response to the one or more utterances; and
communicating, by the digital assistant, the response to the user.
2 . The computer implemented method of claim 1 , wherein the noise comprises one or more spelling mistake, a randomly inserted word, a randomly deleted word, swapped word positions, word synonyms, different word forms, irrelevant words, slang, translations, or any combination thereof.
3 . The computer implemented method of claim 1 , wherein the one of the examples in each pair of modified examples are modified by selecting one or more letters or words from the other of the examples in each respective pair of modified examples, and performing an insert, delete, or replace operation on the one or more letters or words.
4 . The computer implemented method of claim 1 , wherein the pairs of modified examples are generated by attacking the machine-learning model with initial pairs of modified examples, wherein the attacking comprises:
inputting the initial pairs of modified examples into the machine-learning model, generating, by the machine-learning model, a pair of predictions for each of the initial pairs of modified examples,
comparing the pair of predictions for each of the initial pairs of modified examples to determine whether the pair of predictions are the same or different for each of the initial pairs of modified examples; and
when a pair of predictions are different for a given initial pair of modified examples, adding the given initial pair of modified examples to the adversarial examples, which are used for fine-tuning the machine-learning model.
5 . The computer implemented method of claim 4 , wherein the fine-tuning comprises feeding the adversarial examples to the machine-learning model and learning or updating plurality model parameters to account for characteristics of the adversarial examples based on maximizing or minimizing an objective function.
6 . The computer implemented method of claim 4 , wherein the attacking further comprises when the pair of predictions are different and prior to adding the given initial pair of modified examples to the adversarial examples, determining individually whether the modified example of the given initial pair of modified examples is a true adversarial example, and limiting the addition of the modified example to the adversarial examples as only being the addition of the modified example deemed to be a true adversarial example to the adversarial examples.
7 . A system comprising:
one or more processors; and
a memory coupled to the one or more processors, the memory storing a plurality of instructions executable by the one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to perform processing comprising:
receiving, at a digital assistant comprising a chatbot, one or more utterances from a user, wherein the one or more utterances comprise at least one utterance having noise;
inputting the one or more utterances into the chatbot;
processing, by the chatbot, the one or more utterances, wherein the processing comprises:
performing, by a machine learning model, one or more natural language tasks on the one or more utterances to obtain output, wherein the machine learning model is fine-tuned on adversarial examples in order to perform the one or more natural language tasks on utterances that have noise, wherein:
the adversarial examples comprise pairs of modified examples,
one of the examples in each pair of modified examples has been modified from the other of the examples in each respective pair of modified examples in a utility preserving manner,
the utility preserving manner includes modifying the example without changing an original meaning or expected label of the example, and
the pair of modified examples share a same expected label, and
generating, based on the output, a natural language response to the one or more utterances; and
communicating, by the digital assistant, the response to the user.
8 . The system of claim 7 , wherein the noise comprises one or more spelling mistake, a randomly inserted word, a randomly deleted word, swapped word positions, word synonyms, different word forms, irrelevant words, slang, translations, or any combination thereof.
9 . The system of claim 7 , wherein the one of the examples in each pair of modified examples are modified by selecting one or more letters or words from the other of the examples in each respective pair of modified examples, and performing an insert, delete, or replace operation on the one or more letters or words.
10 . The system of claim 8 , wherein the pairs of modified examples are generated by attacking the machine-learning model with initial pairs of modified examples, wherein the attacking comprises:
inputting the initial pairs of modified examples into the machine-learning model, generating, by the machine-learning model, a pair of predictions for each of the initial pairs of modified examples,
comparing the pair of predictions for each of the initial pairs of modified examples to determine whether the pair of predictions are the same or different for each of the initial pairs of modified examples; and
when a pair of predictions are different for a given initial pair of modified examples, adding the given initial pair of modified examples to the adversarial examples, which are used for fine-tuning the machine-learning model.
11 . The system of claim 10 , wherein the fine-tuning comprises feeding the adversarial examples to the machine-learning model and learning or updating plurality model parameters to account for characteristics of the adversarial examples based on maximizing or minimizing an objective function.
12 . The system of claim 10 , wherein the attacking further comprises when the pair of predictions are different and prior to adding the given initial pair of modified examples to the adversarial examples, determining individually whether the modified example of the given initial pair of modified examples is a true adversarial example, and limiting the addition of the modified example to the adversarial examples as only being the addition of the modified example deemed to be a true adversarial example to the adversarial examples.
13 . A non-transitory computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to perform processing comprising:
receiving, at a digital assistant comprising a chatbot, one or more utterances from a user, wherein the one or more utterances comprise at least one utterance having noise;
inputting the one or more utterances into the chatbot;
processing, by the chatbot, the one or more utterances, wherein the processing comprises:
performing, by a machine learning model, one or more natural language tasks on the one or more utterances to obtain output, wherein the machine learning model is fine-tuned on adversarial examples in order to perform the one or more natural language tasks on utterances that have noise, wherein:
the adversarial examples comprise pairs of modified examples,
one of the examples in each pair of modified examples has been modified from the other of the examples in each respective pair of modified examples in a utility preserving manner,
the utility preserving manner includes modifying the example without changing an original meaning or expected label of the example, and
the pair of modified examples share a same expected label, and
generating, based on the output, a natural language response to the one or more utterances; and
communicating, by the digital assistant, the response to the user.
14 . The non-transitory computer-readable memory of claim 13 , wherein the noise comprises one or more spelling mistake, a randomly inserted word, a randomly deleted word, swapped word positions, word synonyms, different word forms, irrelevant words, slang, translations, or any combination thereof.
15 . The non-transitory computer-readable memory of claim 13 , wherein:
the one of the examples in each pair of modified examples are modified by selecting one or more letters or words from the other of the examples in each respective pair of modified examples, and performing an insert, delete, or replace operation on the one or more letters or words.
16 . The non-transitory computer-readable memory of claim 13 , wherein the pairs of modified examples are generated by attacking the machine-learning model with initial pairs of modified examples, wherein the attacking comprises:
inputting the initial pairs of modified examples into the machine-learning model, generating, by the machine-learning model, a pair of predictions for each of the initial pairs of modified examples,
comparing the pair of predictions for each of the initial pairs of modified examples to determine whether the pair of predictions are the same or different for each of the initial pairs of modified examples; and
when a pair of predictions are different for a given initial pair of modified examples, adding the given initial pair of modified examples to the adversarial examples, which are used for fine-tuning the machine-learning model.
17 . The non-transitory computer-readable memory of claim 16 , wherein the fine-tuning comprises feeding the adversarial examples to the machine-learning model and learning or updating plurality model parameters to account for characteristics of the adversarial examples based on maximizing or minimizing an objective function.
18 . The non-transitory computer-readable memory of claim 16 , wherein the attacking further comprises when the pair of predictions are different and prior to adding the given initial pair of modified examples to the adversarial examples, determining individually whether the modified example of the given initial pair of modified examples is a true adversarial example, and limiting the addition of the modified example to the adversarial examples as only being the addition of the modified example deemed to be a true adversarial example to the adversarial examples.