Region constrained regularized adversarial examples for model interpretability

1. A method, implemented in a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method comprising:

receiving, at the computer system, input data and a machine learning model to generate a prediction based on the input data;

generating, at the computer system, a mask indicating portions of the input data to be disregarded during backpropagation of the machine learning model;

modifying, at the computer system, the generated mask to, at least one of:

improve the explanation of the machine learning model, or,

improve the localization of the object classified by the learning model;

wherein, modifying the mask further comprises iteratively modifying, at the computer system, a perturbed or adversarial version of the input data using a gradient of a loss function with respect to the input data, while keeping the machine learning model frozen, wherein the perturbed or adversarial version of the input data is unchanged in a portion defined by the mask; and

generating, at the computer system, a saliency mask approximating support of the perturbation to smoothly approximate a step function.

2. The method of claim 1 , wherein modifying the mask further comprises:

expanding, at the computer system, the mask by adding the generated saliency mask to the mask to generate a modified mask.

3. The method of claim 2 , wherein the loss function loss is based on a first term that reduces a classification value of a class of the machine learning model to a given value, a second term that approximates the size of support of the perturbation, and a third term that encourages smoothness of support of the perturbation, preferring continuous regions of non-zero values over scattered individual elements.

4. The method of claim 3 , wherein the input data is an image.

5. The method of claim 4 , wherein the mask is a portion of the pixels of the image.

6. A system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform:

receiving input data and a machine learning model to generate a prediction based on the input data;

generating a mask indicating portions of the input data to be disregarded during backpropagation of the machine learning model;

modifying the generated mask to, at least one of:

improve the explanation of the machine learning model, or,

improve the localization of the object classified by the learning model;

wherein, modifying the mask further comprises iteratively modifying a perturbed or adversarial version of the input data using a gradient of a loss function with respect to the input data, while keeping the machine learning model frozen, wherein the perturbed or adversarial version of the input data is unchanged in a portion defined by the mask; and

generating a saliency mask approximating support of the perturbation to smoothly approximate a step function.

7. The system of claim 6 , wherein modifying the mask further comprises:

expanding the mask by adding the generated saliency mask to the mask to generate a modified mask.

8. The system of claim 7 , wherein the loss function loss is based on a first term that reduces a classification value of a class of the machine learning model to a given value, a second term that approximates the size of support of the perturbation, and a third term that encourages smoothness of support of the perturbation, preferring continuous regions of non-zero values over scattered individual elements.

9. The system of claim 8 , wherein the input data is an image.

10. The system of claim 9 , wherein the mask is a portion of the pixels of the image.

11. A computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer system, to cause the computer system to perform a method comprising:

receiving, at the computer system, input data and a machine learning model to generate a prediction based on the input data;

generating, at the computer system, a mask indicating portions of the input data to be disregarded during backpropagation of the machine learning model;

modifying, at the computer system, the generated mask to, at least one of:

improve the explanation of the machine learning model, or,

improve the localization of the object classified by the learning model;

wherein, modifying the mask further comprises iteratively modifying, at the computer system, a perturbed or adversarial version of the input data using a gradient of a loss function with respect to the input data, while keeping the machine learning model frozen, wherein the perturbed or adversarial version of the input data is unchanged in a portion defined by the mask; and

generating, at the computer system, a saliency mask approximating support of the perturbation to smoothly approximate a step function.

12. The computer program product of claim 11 , wherein modifying the mask further comprises:

expanding, at the computer system, the mask by adding the generated saliency mask to the mask to generate a modified mask.

13. The computer program product of claim 12 , wherein the loss function loss is based on a first term that reduces a classification value of a class of the machine learning model to a given value, a second term that approximates the size of support of the perturbation, and a third term that encourages smoothness of support of the perturbation, preferring continuous regions of non-zero values over scattered individual elements.

14. The computer program product of claim 13 , wherein the input data is an image.

15. The computer program product of claim 14 , wherein the mask is a portion of the pixels of the image.