Detecting and delaying effect of machine learning model attacks
One embodiment provides a method for delaying malicious attacks on machine learning models that a trained using input captured from a plurality of users, including: deploying a model, said model designed to be used with an application, for responding to requests received from users, wherein the model comprises a machine learning model that has been previously trained using a data set; receiving input from one or more users; determining, using a malicious input detection technique, if the received input comprises malicious input; if the received input comprises malicious input, removing the malicious input from the input to be used to retrain the model; retraining the model using received input that is determined to not be malicious input; and providing, using the retrained model, a response to a received user query, the retrained model delaying the effect of malicious input on provided responses by removing malicious input from retraining input.
1. A method for delaying the effect of malicious attacks on a machine learning model that is continuously retrained using input captured from a plurality of users, comprising:
deploying a model, said model designed to be used with an application, for responding to requests received from users, wherein the model comprises a machine learning model that has been previously trained using a data set;
receiving input from one or more users, wherein the input is queued for retraining the machine learning model;
determining, using a malicious input detection technique, the received input comprises malicious input, wherein the malicious input detection technique at least comprises a temporal detection technique, wherein the temporal detection technique comprises (i) identifying a time period, (ii) identifying the number of inputs received during the identified time period, (iii) determining a nature of the input received during the identified time period, and (iv) classifying the received input as malicious if the identified number of inputs received during the identified time period exceeds a predetermined threshold and the received inputs are determined to have an extreme nature, wherein the predetermined threshold comprises a threshold corresponding to a usual rate of received user input for a predetermined time period similar to the identified time period;
removing the malicious input from input queued for retraining the model;
retraining the model using the input queued for retraining the model without the malicious input; and
providing, using the retrained model, a response to a received user query, the retrained model delaying the effect of malicious input on provided responses by removing malicious input from the input queued for retraining the model.
2. The method of claim 1 , wherein the malicious input detection technique further comprises at least one additional technique selected from the group consisting of: a label detection technique, a user detection technique, and a collusion detection technique.
3. The method of claim 2 , wherein the at least one additional technique comprises a label detection technique comprising (i) identifying a current label corresponding to the application, (ii) determining the received input comprises a label outside a predetermined threshold from the current label, and (iii) classifying the received input as malicious.
4. The method of claim 2 , wherein the at least one additional technique comprises a user detection technique comprising (i) accessing historical information regarding a user providing the received input and (ii) classifying the received input as malicious if the user is identified as having provided past malicious input.
5. The method of claim 2 , wherein the at least one additional technique comprises a collusion detection technique comprising (i) accessing historical information regarding a plurality of users providing the received input and (ii) classifying the received input as malicious if the plurality of users are identified as having previously colluded to provide malicious input.
6. The method of claim 1 , comprising correcting the model by resetting the model to a state previously identified as a clean state comprising a retrained model state known to have not been retrained using malicious input.
7. The method of claim 6 , wherein the correcting is responsive to identifying the model was retrained using malicious input.
8. An apparatus for delaying the effect of malicious attacks on a machine learning model that is continuously retrained using input captured from a plurality of users, comprising:
at least one processor; and
a computer readable storage medium having computer readable program code embodied therewith and executable by the at least one processor, the computer readable program code comprising:
computer readable program code configured to deploy a model, said model designed to be used with an application, for responding to requests received from users, wherein the model comprises a machine learning model that has been previously trained using a data set;
computer readable program code configured to receive input from one or more users, wherein the input is queued for retraining the machine learning model;
computer readable program code configured to determine, using a malicious input detection technique, the received input comprises malicious input, wherein the malicious input detection technique at least comprises a temporal detection technique, wherein the temporal detection technique comprises (i) identifying a time period, (ii) identifying the number of inputs received during the identified time period, (iii) determining a nature of the input received during the identified time period, and (iv) classifying the received input as malicious if the identified number of inputs received during the identified time period exceeds a predetermined threshold and the received inputs are determined to have an extreme nature, wherein the predetermined threshold comprises a threshold corresponding to a usual rate of received user input for a predetermined time period similar to the identified time period;
computer readable program code configured to remove the malicious input from input queued for retraining the model;
computer readable program code configured to retrain the model using the input queued for retraining the model without the malicious input; and
computer readable program code configured to provide, using the retrained model, a response to a received user query, the retrained model delaying the effect of malicious input on provided responses by removing malicious input from the input queued for retraining the model.
9. A computer program product for delaying the effect of malicious attacks on a machine learning model that is continuously retrained using input captured from a plurality of users, comprising:
a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code executable by a processor and comprising:
computer readable program code configured to deploy a model, said model designed to be used with an application, for responding to requests received from users, wherein the model comprises a machine learning model that has been previously trained using a data set;
computer readable program code configured to receive input from one or more users, wherein the input is queued for retraining the machine learning model;
computer readable program code configured to determine, using a malicious input detection technique, the received input comprises malicious input, wherein the malicious input detection technique at least comprises a temporal detection technique, wherein the temporal detection technique comprises (i) identifying a time period, (ii) identifying the number of inputs received during the identified time period, (iii) determining a nature of the input received during the identified time period, and (iv) classifying the received input as malicious if the identified number of inputs received during the identified time period exceeds a predetermined threshold and the received inputs are determined to have an extreme nature, wherein the predetermined threshold comprises a threshold corresponding to a usual rate of received user input for a predetermined time period similar to the identified time period;
computer readable program code configured to remove the malicious input from input queued for retraining the model;
computer readable program code configured to retrain the model using the input queued for retraining the model without the malicious input; and
computer readable program code configured to provide, using the retrained model, a response to a received user query, the retrained model delaying the effect of malicious input on provided responses by removing malicious input from the input queued for retraining the model.
10. The computer program product of claim 9 , wherein the malicious input detection technique further comprises a label detection technique comprising (i) identifying a current label corresponding to the application, (ii) determining the received input comprises a label outside a predetermined threshold from the current label, and (iii) classifying the received input as malicious.
11. The computer program product of claim 9 , wherein the malicious input detection technique further comprises a user detection technique comprising (i) accessing historical information regarding a user providing the received input and (ii) classifying the received input as malicious if the user is identified as having provided past malicious input.
12. The computer program product of claim 9 , wherein the malicious input detection technique further comprises a collusion detection technique comprising (i) accessing historical information regarding a plurality of users providing the received input and (ii) classifying the received input as malicious if the plurality of users are identified as having previously colluded to provide malicious input.
13. The computer program product of claim 9 , comprising correcting the model by resetting the model to a state previously identified as a clean state comprising a retrained model state known to have not been retrained using malicious input.