Resiliency of machine learning models
One embodiment provides a method, including: deploying a machine learning model, wherein the deployed machine learning model is used in responding to queries from users; receiving, at the deployed machine learning model, input from a user; identifying a type of machine learning model attack corresponding to the received input; computing, responsive to receiving the input, a resiliency score of the machine learning model, wherein the resiliency score indicates resistance of the machine learning model against the identified type of attack; and performing an action responsive to the computed resiliency score.
1. A method, comprising:
deploying a machine learning model, wherein the deployed machine learning model is used in responding to queries from users;
receiving, at the deployed machine learning model, input from a user;
identifying a type of machine learning model attack corresponding to the received input;
computing, responsive to receiving the input, a resiliency score of the machine learning model, wherein the resiliency score indicates resistance of the machine learning model against the identified type of attack; and
performing an action responsive to the computed resiliency score.
2. The method of claim 1 , wherein the received input comprises feedback, and wherein the identified type of machine learning model attack comprises a retraining attack.
3. The method of claim 2 , wherein the resiliency score corresponding to the retraining attack is based upon at least one of: (i) the stability of the machine learning model, (ii) the regularization of the machine learning model, (iii) the number of training parameters of the machine learning model, and (iv) the training data distribution of the machine learning model.
4. The method of claim 1 , wherein the received input comprises a user query, and wherein the identified type of machine learning model attack comprises a stealing attack.
5. The method of claim 4 , wherein the resiliency score corresponding to the stealing attack is based upon at least one of: (i) the number of parameters of the model, (ii) the number of query samples needed to learn the machine learning model, and (iii) the type of queries needed to learn the machine learning model.
6. The method of claim 1 , wherein the resiliency score comprises (i) a component identifying the resistance of the machine learning model against retraining attacks and (ii) a component identifying the resistance of the machine learning model against stealing attacks.
7. The method of claim 1 , wherein the computing a resiliency score comprises computing an overall resiliency score indicating resistance of the machine learning model against all attack types.
8. The method of claim 1 , comprising identifying a component of the machine learning model contributing to a machine learning model having a computed resiliency score below a predetermined threshold.
9. The method of claim 1 , wherein the performing an action comprises notifying a user if the computed resiliency score is below a predetermined threshold.
10. The method of claim 1 , wherein the performing an action comprises performing counter measures in response to the received input.
11. An apparatus, comprising:
at least one processor; and
a computer readable storage medium having computer readable program code embodied therewith and executable by the at least one processor, the computer readable program code comprising:
computer readable program code configured to deploy a machine learning model, wherein the deployed machine learning model is used in responding to queries from users;
computer readable program code configured to receive, at the deployed machine learning model, input from a user;
computer readable program code configured to identify a type of machine learning model attack corresponding to the received input;
computer readable program code configured to compute, responsive to receiving the input, a resiliency score of the machine learning model, wherein the resiliency score indicates resistance of the machine learning model against the identified type of attack; and
computer readable program code configured to perform an action responsive to the computed resiliency score.
12. A computer program product, comprising:
a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code executable by a processor and comprising:
computer readable program code configured to deploy a machine learning model, wherein the deployed machine learning model is used in responding to queries from users;
computer readable program code configured to receive, at the deployed machine learning model, input from a user;
computer readable program code configured to identify a type of machine learning model attack corresponding to the received input;
computer readable program code configured to compute, responsive to receiving the input, a resiliency score of the machine learning model, wherein the resiliency score indicates resistance of the machine learning model against the identified type of attack; and
computer readable program code configured to perform an action responsive to the computed resiliency score.
13. The computer program product of claim 12 , wherein the received input comprises feedback; and
wherein the identified type of machine learning model attack comprises a retraining attack; and
wherein the resiliency score corresponding to the retraining attack is based upon at least one of: (i) the stability of the machine learning model, (ii) the regularization of the machine learning model, (iii) the number of training parameters of the machine learning model, and (iv) the training data distribution of the machine learning model.
14. The computer program product of claim 12 , wherein the received input comprises a user query; and
wherein the identified type of machine learning model attack comprises a stealing attack; and
wherein the resiliency score corresponding to the stealing attack is based upon at least one of: (i) the number of parameters of the model, (ii) the number of query samples needed to learn the machine learning model, and (iii) the type of queries needed to learn the machine learning model.
15. The computer program product of claim 12 , wherein the resiliency score comprises (i) a component identifying the resistance of the machine learning model against retraining attacks and (ii) a component identifying the resistance of the machine learning model against stealing attacks.
16. The computer program product of claim 12 , wherein the computing a resiliency score comprises computing an overall resiliency score indicating resistance of the machine learning model against all attack types.
17. The computer program product of claim 12 , comprising identifying a component of the machine learning model contributing to a machine learning model having a computed resiliency score below a predetermined threshold.
18. The computer program product of claim 12 , wherein the performing an action comprises notifying a user if the computed resiliency score is below a predetermined threshold.
19. The computer program product of claim 12 , wherein the performing an action comprises performing counter measures in response to the received input.
20. A method, comprising:
employing a machine learning model to respond to queries from one or more users;
receiving, at the machine learning model, input from one or more users, wherein the received input comprises at least one of: machine learning model training input and a query;
calculating a strength score for the machine learning model, wherein the strength score indicates the strength of the machine learning model against an attack from one or more users through received input and wherein the attack is based upon the received input; and
providing a notification to a user if the calculated strength score is below a predetermined threshold.