Testing adversarial robustness of systems with limited access
An adversarial robustness testing method, system, and computer program product include testing a robustness of a black-box system under different access settings via an accelerator.
1. A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings,
wherein the accelerator includes a function to reduce an attack space in the adversarial attack for query efficiency,
wherein a robustness objective for the testing of the robustness of the black-box system uses system defined threat models for adversarial examples, and
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between the adversarial examples and a legitimate image.
2. The method of claim 1 , wherein the different access settings comprise:
a soft-label setting; and
a hard-label setting.
3. The method of claim 1 , further comprising:
for a soft-label setting as one of the different access settings, using the accelerator and a gradient descent technique to find the adversarial examples and summarize a robustness statistic; and
for a hard-label setting as one of the different access settings, using a smoothing function to summarize a robustness statistic.
4. The method of claim 1 , further comprising, given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using the accelerator such that the perturbed example is misclassified to a target class including an incorrect class label by a deep neural network (DNN) model trained on the legitimate inputs.
5. The method of claim 1 , wherein the accelerator comprises a function including an efficient gradient estimation via a random directional estimate and averaging.
6. The method of claim 1 , wherein the accelerator comprises a function including a dimension reduction of an input.
7. The method of claim 1 , wherein the accelerator comprises a function including a problem splitting between a black-box loss function and a white-box adversarial distortion function.
8. The method of claim 1 , embodied in a cloud-computing environment.
9. A computer program product for adversarial robustness testing for checking a learning performance of a black-box system, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:
testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings,
wherein the accelerator includes a function to reduce an attack space in the adversarial attack for query efficiency,
wherein a robustness objective for the testing of the robustness of the black-box system uses system defined threat models for adversarial examples, and
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between the adversarial examples and the legitimate image.
10. The computer program product of claim 9 , wherein the different access settings comprise:
a soft-label setting; and
a hard-label setting.
11. The computer program product of claim 9 , further comprising:
for a soft-label setting as one of the different access settings, using the accelerator and a gradient descent technique to find adversarial example and summarize a robustness statistic; and
for a hard-label setting as one of the different access settings, using a smoothing function to summarize a robustness statistic.
12. The computer program product of claim 9 , further comprising, given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using the accelerator such that the perturbed example is misclassified to a target class including an incorrect class label by a deep neural network (DNN) model trained on the legitimate inputs.
13. The computer program product of claim 9 , wherein the accelerator comprises a function including an efficient gradient estimation via a random directional estimate and averaging.
14. The computer program product of claim 9 , wherein the accelerator comprises a function including a dimension reduction of an input.
15. The computer program product of claim 9 , wherein the accelerator comprises a function including a problem splitting between a black-box loss function and a white-box adversarial distortion function.
16. An adversarial robustness testing system for checking a learning performance of a black-box system, the system comprising:
a processor; and
a memory, the memory storing instructions to cause the processor to perform:
testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings,
wherein the accelerator includes a function to reduce an attack space in the adversarial attack for query efficiency,
wherein a robustness objective for the testing of the robustness of the black-box system uses system defined threat models for adversarial examples, and
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between the adversarial examples and the legitimate image.
17. The system of claim 16 , further comprising:
for a soft-label setting as one of the different access settings, using the accelerator and a gradient descent technique to find the adversarial examples and summarize a robustness statistic; and
for a hard-label setting as one of the different access settings, using a smoothing function to summarize a robustness statistic.
18. A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
testing a robustness, against an adversarial attack, of the black-box system under a limited access setting to the black-box system:
receiving a first classification of an input as an output from the black-box system; and
determining a minimal change to the input such that a second classification is received as the output from the black-box system,
wherein the testing includes a function to reduce an attack space in the adversarial attack for query efficiency,
wherein a robustness objective for the testing of the robustness of the black-box system uses system defined threat models for adversarial examples, and
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between the adversarial examples and the legitimate image.
19. A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
testing a robustness, against an adversarial attack, of the black-box system under a limited access setting to the black-box system:
finding a minimal change to an input such that a classification of an output from the black-box system is different than an original classification of the input, wherein the testing includes a function to reduce an attack space in the adversarial attack for query efficiency,
wherein a robustness objective for the testing of the robustness of the black-box system uses system defined threat models for adversarial examples, and
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between the adversarial examples and the legitimate image.
20. The computer-implemented adversarial robustness testing method of claim 19 , further comprising implementing the minimal change to the input.
21. The method of claim 1 , wherein the limited access setting includes internal structures and configurations that are unknown to the adversary.
22. The method of claim 1 , wherein the testing the robustness of the black-box system includes:
given a legitimate image with a correct class label, designing an optimal adversarial perturbation for the adversarial examples so that the perturbed example is misclassified to a target class by a DNN model trained on legitimate images.
23. The method of claim 1 , wherein the function to reduce the attack space includes only using one step of a gradient estimation, thereby increasing query efficiency.