Framework for certifying a lower bound on a robustness level of convolutional neural networks
A certification method, system, and computer program product include certifying an adversarial robustness of a convolutional neural network by deriving an analytic solution for a neural network output using an efficient upper bound and an efficient lower bound on an activation function and applying the analytic solution in computing a certified robustness.
1. A computer-implemented certification method, the method comprising:
certifying a numerical level of robustness of various architectures of convolutional neutral networks (CNNs) with minimum adversarial distortion by:
deriving an analytic solution for a neural network output of the CNNs using an efficient block-wise linear bound on an activation function separately on non-linear operations in the CNNs,
wherein the efficient bound is derived using convolution operations.
2. The method of claim 1 , further comprising varying the activation function.
3. The method of claim 2 , wherein the activation function is varied until a numerical level of robustness of the neural network is within a predetermined threshold value.
4. The method of claim 1 , further comprising varying building blocks in the neural network.
5. The method of claim 4 , wherein the activation function is varied until a numerical level of robustness of the neural network is within a predetermined threshold value.
6. The method of claim 1 , further comprising varying both of the activation function and building blocks in the neural network.
7. The method of claim 1 , wherein the analytic solution is applied with a binary search.
8. The method of claim 1 , wherein an efficient upper bound as one of the efficient bound comprises a linear upper bound, and
wherein an efficient lower bound as one of the efficient bound comprises a linear lower bound.
9. The method of claim 1 , wherein the adversarial robustness is certified for a same input.
10. The method of claim 1 , embodied in a cloud-computing environment.
11. The method of claim 1 , further comprising computing the numerical level of the robustness a specific architecture of the CNNs based on the analytic solution,
wherein the analytic solution includes deriving, for each building block in the form of element-wise inequality equations, and then plugging in the corresponding bounds and back-propagate to a previous layer of the CNNs.
12. A computer program product for certification, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:
certifying a numerical level of robustness of various architectures of convolutional neutral networks (CNNs) with minimum adversarial distortion by:
deriving an analytic solution for a neural network output of the CNNs using an efficient block-wise linear bound on an activation function separately on non-linear operations in the CNNs,
wherein the efficient bound is derived using convolution operations.
13. The computer program product of claim 12 , further comprising varying the activation function.
14. The computer program product of claim 13 , wherein the activation function is varied until a certified robustness of the neural network is within a predetermined threshold value.
15. The computer program product of claim 12 , further comprising varying building blocks in the neural network.
16. The computer program product of claim 15 , wherein the activation function is varied until a certified robustness of the neural network is within a predetermined threshold value.
17. The computer program product of claim 12 , further comprising varying both of the activation function and building blocks in the neural network.
18. The computer program product of claim 12 , wherein the analytic solution is applied with a binary search.
19. The computer program product of claim 12 , wherein an efficient upper bound as one of the efficient bound comprises a linear upper bound, and
wherein an efficient lower bound as one of the efficient bound comprises a linear lower bound.
20. A certification system, the system comprising:
a processor, and
a memory, the memory storing instructions to cause the processor to perform:
certifying a numerical level of robustness of various architectures of convolutional neutral networks (CNNs) with minimum adversarial distortion by:
deriving an analytic solution for a neural network output of the CNNs using an efficient block-wise linear bound on an activation function separately on non-linear operations in the CNNs,
wherein the efficient bound is derived using convolution operations.