System and method for estimating perturbation norm for the spectrum of robustness
View Patent ↗A computer-program product storing instructions which, when executed by a computer, cause the computer to, for one or more iterations, update parameters associated with a machine-learning network utilizing perturbations for input data, wherein the perturbations are sampled utilizing Markov chain Monte Carlo, identify a loss value associated with each perturbation in each iteration, and evaluate the machine learning network by identifying an average loss value across each iteration and outputting the average loss value.
1 . A computer-implemented method for training a neural network, comprising:
receiving a set of input data from one or more sensors, wherein the data includes time-series data, image data, video data, or sound data;
initializing a random perturbation sample associated with the set of input data;
for one or more iterations, executing one or more steps including:
computing a loss value associated with the random perturbation sample associated with the set of input data;
determining a gradient of the loss value associated with one or more parameters of the neural network;
updating, utilizing one or more machine learning optimizers, the one or more parameters in response to the gradient of the loss value associated with an intermediate-p robustness, wherein the intermediate-p robustness is an expectation of a p-norm of a loss, where 1<p<∞; increasing a level of perturbation stress applied to a new perturbation for one or more successive iterations; and
in response to exceeding a first threshold associated with convergence of the neural network, outputting a trained neural network utilizing the updated parameters.
2 . The computer-implemented method of claim 1 , wherein the method includes utilizing Markov chain Monte Carlo to initialize the random perturbation sample.
3 . The computer-implemented method of claim 1 , wherein the neural network is further configured to execute a plurality of iterations of updating the one or more parameters utilizing the gradient prior to outputting the trained neural network.
4 . The computer-implemented method of claim 1 , wherein the trained neural network is configured to identify a random perturbation.
5 . The computer-implemented method of claim 1 , wherein a density associated with the random perturbation sample is a uniform distribution over a norm ball.
6 . The computer-implemented method of claim 1 , wherein the first threshold includes an amount of loss of the input data.
7 . The computer-implemented method of claim 1 , wherein the method includes increasing a level of perturbation stress for the random perturbation sample.
8 . The computer-implemented method of claim 1 , wherein the method includes utilizing perturbations associated with the set of the input data, wherein the perturbations are sampled utilizing Markov chain Monte Carlo for one or more iterations.
9 . The computer-implemented method of claim 1 , wherein the method includes evaluating an intermediate-p robustness for one or more perturbations utilizing an estimator.
10 . A system including a machine-learning network, comprising:
a data storage interface configured to receive input data from a sensor, wherein the sensor includes a camera, a radar, a sonar, or a microphone;
a processor, in communication with the data storage interface, wherein the processor is programmed to:
receive the input data, wherein the input data is indicative of image, radar, sonar, or sound information;
initiate a random perturbation sample associated with the input data, wherein the random perturbation sample is derived from path sampling;
for one or more iterations, executing one or more steps including:
increase a level of perturbation stress for the random perturbation sample;
compute a loss value associated with the random perturbation sample;
determine a gradient of the loss value associated with one or more parameters of the neural network;
update the one or more parameters utilizing the gradient;
increasing a level of perturbation stress applied to a new perturbation to be utilized;
increasing a level and evaluate an intermediate-p robustness for one or more perturbations utilizing an estimator, wherein the intermediate-p robustness is an expectation of a p-norm of a loss, where 1<p<∞; and in response to exceeding a first threshold associated with convergence of the neural network, output a trained neural network utilizing updated parameters.
11 . The system of claim 10 , wherein the level of perturbation stress for the random perturbation sample is adjusted at each iteration.
12 . The system of claim 10 , wherein the level of perturbation stress for the random perturbation sample is increased at each iteration.
13 . The system of claim 10 , wherein the perturbations are sampled utilizing Markov chain Monte Carlo for one or more iterations.
14 . The system of claim 10 , wherein the first threshold includes a number of iterations.
15 . A computer-program product in a computer comprising a non-transitory computer readable storage medium storing instruction, that when executed by the computer, cause the computer to:
for one or more iterations, update parameters associated with a machine-learning network utilizing perturbations for input data received from a sensor, wherein the sensor includes a camera, a radar, a sonar, or a microphone, and wherein the perturbations are sampled utilizing Markov chain Monte Carlo;
identify a loss value associated with each perturbation in each iteration;
evaluate the machine learning network by identifying an average loss value across each iteration and outputting the average loss value;
determine a gradient of the loss value associated with one or more parameters of the machine-learning network, the gradient of the loss value associated with an intermediate-p robustness, wherein the intermediate-p robustness is an expectation of a p-norm of a loss, where 1<p<∞; update the one or more parameters utilizing the gradient; and
output a trained machine-learning network utilizing updated parameters and upon convergence to a first threshold.
16 . The computer-program product in the computer of claim 15 , wherein the machine-learning network is further configured to execute a plurality of iterations of updating the one or more parameters utilizing the gradient prior to outputting the trained machine-learning network.
17 . The computer-program product in the computer of claim 15 , wherein the trained machine-learning model is configured to identify a random perturbation.
18 . The computer-program product in the computer of claim 15 , wherein the first threshold includes an amount of loss of the input data.
19 . The computer-program product in the computer of claim 15 , wherein the first threshold includes a number of iterations.