System and method for query efficient blackbox physically realizable attack with bayesian optimization
A system includes a machine learning network input interface configured to receive input data from a sensor, one or more processors collectively programmed to receive an input data from the sensor, wherein the input data is indicative of image of a scene that includes a perturbation from a black-box attack with a physical perturbation at the scene, display an adversarial pattern at the scene, determine an objective function utilizing at least the adversarial pattern and a target classification of the machine-learning network, randomly select a plurality of data points associated with the adversarial pattern and the objective function, wherein the data points are associated with a number of queries of the objective function, obtain a machine-learning model output utilizing the data points displayed in the scene, and in response to meeting a criteria associated with the adversarial pattern and model output, identify a successful attack pattern.
1 . A computer-implemented method for attacking a machine-learning model, comprising:
determine an objective function utilizing at least an adversarial pattern displayed at a scene, and either (1) output of the machine-learning network, or (2) a target output of the machine-learning network;
determining a maximum number of queries to the machine learning model, which includes a number of initial queries,
randomly selecting a plurality of data points within the adversarial pattern's space and the objective function, wherein the plurality of data points are associated with a number of initial queries;
displaying the plurality of data points in the scene to generate a rendered scene utilizing a system, wherein the rendered scene is received as input for the machine-learning model;
obtaining a machine-learning model output utilizing the randomly selected plurality of data points displayed at the rendered scene;
in response to the corresponding objective values given the previous plurality of data points and the machine-learning output, creating the next adversarial pattern at the system,
displaying the next adversarial pattern and querying the machine learning model; and
generating, via the system, additional adversarial patterns until identifying a successful attack pattern meeting a success criteria associated with the adversarial pattern and the machine-learning model output, or the maximum number of queries has reached.
2 . The computer-implemented method of claim 1 , wherein the dimensional input is 5×5.
3 . The computer-implemented method of claim 1 , wherein the adversarial pattern is an RGB image.
4 . The computer-implemented method of claim 1 , update the adversarial pattern with Bayesian optimization utilizing the objective function.
5 . The computer-implemented method of claim 1 , wherein no training data is utilized to identify the successful pattern.
6 . The computer-implemented method of claim 1 , wherein the adversarial pattern is located on a display or monitor in a scene within sensor range.
7 . The computer-implemented method of claim 1 , wherein the method includes initializing and updating the adversarial pattern with Bayesian optimization utilizing the objective function.
8 . The computer-implemented method of claim 1 , wherein the input data includes video information obtained from the camera.
9 . A system including an attack for a machine-learning network, comprising:
a machine learning network input interface configured to receive input data from a sensor, wherein the sensor includes a camera;
one or more processors in communication with the input interface, wherein the one or more processors are collectively programmed to:
receive an input data from the sensor, wherein the input data is indicative of image of a scene that includes a perturbation from a black-box attack with a physical perturbation at the scene;
display an adversarial pattern at the scene;
determine an objective function utilizing at least the adversarial pattern and a target classification of the machine-learning network; and
randomly select a plurality of data points associated with the adversarial pattern and the objective function, wherein the plurality of data points are associated with a number of queries of the objective function;
obtain a machine-learning model output utilizing the randomly selected plurality of data points displayed in the scene; and
in response to meeting a success criteria associated with the adversarial pattern and the machine-learning model output, identify a successful attack pattern associated with the adversarial pattern.
10 . The system of claim 9 , wherein the processor is programmed to randomly convert the adversarial pattern utilizing a monitor display function.
11 . The system of claim 9 , wherein the dimensional input is 5×5 dimensions or less.
12 . The system of claim 9 , wherein the one or more processors is a single processor.
13 . The system of claim 9 , wherein the one or more processors are collectively programmed to initialize and update the adversarial pattern with Bayesian optimization utilizing the objective function.
14 . The system of 9 , wherein the input data includes radar, sonar, or sound information.
15 . A computer-implemented method for attacking a machine-learning model, comprising:
receiving input data from one or more sensors and a display in proximity to the sensors,
determining an objective function utilizing at least the adversarial pattern, and the classification of the machine-learning network, or a target classification of the machine-learning network;
determining a maximum number of queries to the machine learning model, which includes a number of initial queries,
randomly selecting a plurality of data points within the adversarial pattern's space and the objective function, wherein the plurality of data points are associated with a number of initial queries;
obtaining a machine-learning model output utilizing the randomly selected plurality of data points converted; and
displaying a plurality of randomly selected data points in a scene, wherein a a rendered scene is received as input for the machine-learning model;
obtaining a machine-learning model output utilizing the randomly selected plurality of data points displayed;
in response to the corresponding objective values given previous plurality of data points and the machine-learning output, creating a next adversarial pattern;
displaying the next adversarial pattern and query the machine-learning model;
creating new adversarial patterns until either (1) identifying a successful attack pattern meeting a criteria associated with the adversarial pattern and the machine-learning model output, or (2) the maximum number of queries has reached.
16 . The computer implemented method of claim 15 , wherein the instructions include selecting the plurality of data points is conducted randomly.
17 . The computer implemented method of claim 15 , wherein the objective function includes a loss between output associated with a target classification and a benign scene.
18 . The computer implemented method of claim 15 , wherein the instructions include initializing and updating the adversarial pattern with Bayesian optimization utilizing the objective function.
19 . The computer implemented method of claim 18 , wherein the Bayesian optimization utilizes a Gaussian process to approximate a joint distribution of samples associated with the adversarial pattern.
20 . The computer implemented method of claim 18 , wherein the success criteria includes a threshold associated with the adversarial pattern.