Method and system for machine learning model testing and preventive measure recommendation
Data-driven applications depend on training data obtained from multiple internal and external data sources. Hence poisoning of the training data can cause adverse effects in the data driven applications. Conventional methods identifies contaminated test samples and avert them from entering into the training. A generic approach covering all data-driven applications and all types of data poisoning attacks in an efficient manner is challenging. Initially, data aggregation is performed after receiving a ML application for testing. A plurality of feature vectors are extracted from the aggregated data and a poisoned data set is generated. A plurality of personas are generated and are further prioritized to obtain a plurality of attack personas. Further, a plurality of security assessment vectors are computed for each of the plurality of attack personas. A plurality of preventive measures are recommended for each of the plurality of attack personas based on the corresponding security assessment vector.
1. A processor implemented method, the method comprising:
receiving, via one or more hardware processors, a Machine Learning (ML) based application to be tested;
aggregating, via the one or more hardware processors, data associated with the ML application to be tested from a plurality of external data sources;
computing, via the one or more hardware processors, a plurality of feature vectors based on the aggregated data by utilizing a plurality of feature engineering techniques, wherein each of the plurality of feature vectors is associated with a corresponding label vector, wherein the corresponding label vector is a significant attribute of a dataset associated with the ML application to be tested;
computing, via the one or more hardware processors, a poisoned dataset based on the plurality of feature vectors by using a label flipping, wherein the poisoned data set is a perturbed sample generated based on an impact of a corresponding feature in the ML based application to be tested;
performing, via the one or more hardware processors, a training of a ML model with the aggregated data and the poisoned dataset;
simultaneously combining, via the one or more hardware processors, a plurality of threat attributes to generate a plurality of personas, wherein each of the plurality of threat attributes is associated with a numeric value;
computing, via the one or more hardware processors, a priority value for each of the plurality of personas based on the numeric value associated with each of the plurality of threat attributes corresponding to each of the plurality of personas by using one of, a summing of the numeric values associated with each plurality of threat attributes, a simple average method and a weighted average method;
sorting, via the one or more hardware processors, the plurality of personas in a descending order based on the corresponding priority value to obtain a plurality of sorted personas;
selecting, via the one or more hardware processors, a plurality of attack personas from the plurality of sorted personas based on a plurality of pre-determined threshold values corresponding to each of a plurality of system constraints associated with the system;
computing, via the one or more hardware processors, a security assessment vector for each of the plurality of attack personas based on a plurality of security evaluation metrics by using the trained ML model, by:
computing, via the ML model trained using the aggregated data, a first set of security evaluation metrics for each of the plurality of attack personas;
computing, via the ML model trained using the aggregated data, a second set of security evaluation metrics for each of the plurality of attack personas; and
computing a difference between each element of the first set of security evaluation metrics and each element of the second set of security evaluation metrics to obtain the security assessment vector corresponding to each of the plurality of attack personas;
recommending, via the one or more hardware processors, a plurality of preventive measures for each of the plurality of attack personas based on the security assessment vector using a ranking method, wherein the ranking method selects the plurality of preventive measures based on a minimum distance between a preventive measure and an attack persona.
2. The processor implemented method of claim 1 , wherein the data comprises textual data, binary data and multimedia data, wherein the data is one of, a structured data, an unstructured data and a semi-structured data, and wherein the plurality of external data sources comprising online and offline data sources.
3. The processor implemented method of claim 1 , wherein the plurality of feature engineering techniques comprising a bag of words, a scalarisation, a normalization, a recoding, and a feature transformation.
4. The processor implemented method of claim 1 , wherein computing the poisoned data set based on the plurality of feature vectors by using the label flipping comprises:
receiving a flip budget, the data, the plurality of feature vectors and the corresponding label vectors, wherein the flip budget is a pre-computed based on a plurality of constraints comprising a number of queries and a type of updation performed by the ML model comprising one of, a batch updation and an online updation;
segmenting the data into a training data and a testing data;
computing a loss function based on the training data, the plurality of feature vectors of the training data and the corresponding label vectors by using an optimization algorithm, wherein the optimization algorithm comprises one of, a gradient based optimization algorithm and a non-gradient based algorithm;
sorting the training data based on the computed loss function;
selecting a plurality of data points from the sorted training data based on the flip budget; and
flipping one or more label vectors of the plurality of selected data points to generate the poisoned data set.
5. The processor implemented method of claim 1 , wherein the plurality of threat attributes comprises an attacker's skill, an attack time constraint, a percentage of data accessible to an attacker, a number of attacker queries, a complexity of an attack, a transferability of the attack, an attacker's knowledge about an ML model, an attacker's reachability to the dataset and an attacker's goal.
6. The processor implemented method of claim 1 , wherein the plurality of system constraints comprises a number of queries, a type of updation performed by the ML model, a network latency, a system load, a query processing time, a processing limit of a data poisoning algorithm, a time delay between successive queries, a percentage of total data size and a size of the dataset.
7. The processor implemented method of claim 1 , wherein the plurality of security evaluation metrics comprises a performance degradation, a False Positive Rate (FPR), a True Positive Rate (TPR), a fraction of compromised components, the attack performance and a susceptibility of the ML model.
8. A system comprising:
at least one memory storing programmed instructions;
one or more Input/Output (I/O) interfaces; and
one or more hardware processors operatively coupled to the at least one memory, wherein the one or more hardware processors are configured by the programmed instructions to:
receive a Machine Learning (ML) based application to be tested;
aggregate data associated with the ML application to be tested from a plurality of external data sources;
compute a plurality of feature vectors based on the aggregated data by utilizing a plurality of feature engineering techniques, wherein each of the plurality of feature vectors is associated with a corresponding label vector, wherein the corresponding label vector is a significant attribute of a dataset associated with the ML application to be tested;
compute a poisoned data set based on the plurality of feature vectors by using a label flipping, wherein the poisoned dataset is a perturbed sample generated based on an impact of a corresponding feature in the ML based application to be tested;
perform via the one or more hardware processors, a training of a ML model with the aggregated data and the poisoned dataset;
simultaneously combine via the one or more hardware processors, a plurality of threat attributes to generate a plurality of personas, wherein each of the plurality of threat attributes is associated with a numeric value;
compute a priority value for each of the plurality of personas based on the numeric value associated with each of the plurality of threat attributes corresponding to each of the plurality of personas by using one of, a summing of the numeric values associated with each plurality of threat attributes, a simple average method and a weighted average method;
sort the plurality of personas in a descending order based on the corresponding priority value to obtain a plurality of sorted personas;
selecting a plurality of attack personas from the plurality of sorted personas based on a plurality of pre-determined threshold values corresponding to each of a plurality of system constraints associated with the system;
compute a security assessment vector for each of the plurality of attack personas based on a plurality of security evaluation metrics by using the trained ML model, by:
computing, via the ML model trained using the aggregated data, a first set of security evaluation metrics for each of the plurality of attack personas;
computing, via the ML model trained using the aggregated data, a second set of security evaluation metrics for each of the plurality of attack personas; and
computing a difference between each element of the first set of security evaluation metrics and each element of the second set of security evaluation metrics to obtain the security assessment vector corresponding to each of the plurality of attack personas;
recommend a plurality of preventive measures for each of the plurality of attack personas based on the security assessment vector using a ranking method, wherein the ranking method selects the plurality of preventive measures based on a minimum distance between a preventive measure and an attack persona.
9. The system of claim 8 , wherein the data comprises textual data, binary data and multimedia data, wherein the data is one of, a structured data, an unstructured data and a semi-structured data, and wherein the plurality of external data sources comprising online and offline data sources.
10. The system of claim 8 , wherein the plurality of feature engineering techniques comprising a bag of words, a scalarisation, a normalization, a recoding and a feature transformation.
11. The system of claim 8 , wherein computing the poisoned data set based on the plurality of feature vectors by using the label flipping comprises:
receiving a flip budget, the data, the plurality of feature vectors and the corresponding label vectors, wherein the flip budget is a pre-computed based on a plurality of constraints comprising a number of queries and a type of updation performed by the ML model comprising one of, a batch updation and an online updation;
segmenting the data into a training data and a testing data;
computing a loss function based on the training data, the plurality of feature vectors of the training data and the corresponding label vectors by using an optimization algorithm, wherein the optimization algorithm comprises one of, a gradient based optimization algorithm and a non-gradient based algorithm;
sorting the training data based on the computed loss function;
selecting a plurality of data points from the sorted training data based on the flip budget; and
flipping one or more label vectors of the plurality of selected data points to generate the poisoned data set.
12. The system of claim 8 , wherein the plurality of threat attributes comprises an attacker's skill, an attack time constraint, a percentage of data accessible to an attacker, a number of attacker queries, a complexity of an attack, a transferability of the attack, an attacker's knowledge about an ML model, an attacker's reachability to the dataset and an attacker's goal.
13. The system of claim 8 , wherein the plurality of system constraints comprises a number of queries, a type of updation performed by the ML model, a network latency, a system load, a query processing time, a processing limit of a data poisoning algorithm, a time delay between successive queries, a percentage of total data size and a size of the dataset.
14. The system of claim 8 , wherein the plurality of security evaluation metrics comprises a performance degradation, a False Positive Rate (FPR), a True Positive Rate (TPR), a fraction of compromised components, the attack performance and a susceptibility of the ML model.
15. A non-transitory computer readable medium embodying a program executable in a computing device for predicting response time of an enterprise system, the program comprising:
receiving a Machine Learning (ML) based application to be tested;
aggregating data associated with the ML application to be tested from a plurality of external data sources;
computing a plurality of feature vectors based on the aggregated data by utilizing a plurality of feature engineering techniques, wherein each of the plurality of feature vectors is associated with a corresponding label vector, wherein the corresponding label vector is a significant attribute of a dataset associated with the ML application to be tested;
computing a poisoned data set based on the plurality of feature vectors by using a label flipping, wherein the poisoned dataset is a perturbed sample generated based on an impact of a corresponding feature in the ML based application to be tested;
performing a training of a ML model with the aggregated data and the poisoned dataset;
simultaneously combining a plurality of threat attributes to generate a plurality of personas, wherein each of the plurality of threat attributes is associated with a numeric value;
computing a priority value for each of the plurality of personas based on the numeric value associated with each of the plurality of threat attributes corresponding to each of the plurality of personas by using one of, a summing of the numeric values associated with each plurality of threat attributes, a simple average method and a weighted average method;
sorting the plurality of personas in a descending order based on the corresponding priority value to obtain a plurality of sorted personas;
selecting a plurality of attack personas from the plurality of sorted personas based on a plurality of pre-determined threshold values corresponding to each of a plurality of system constraints associated with the system;
computing a security assessment vector for each of the plurality of attack personas based on a plurality of security evaluation metrics by using the trained ML model, by:
computing, via the ML model trained using the aggregated data, a first set of security evaluation metrics for each of the plurality of attack personas;
computing, via the ML model trained using the aggregated data, a second set of security evaluation metrics for each of the plurality of attack personas; and
computing a difference between each element of the first set of security evaluation metrics and each element of the second set of security evaluation metrics to obtain the security assessment vector corresponding to each of the plurality of attack personas;
recommending a plurality of preventive measures for each of the plurality of attack personas based on the security assessment vector using a ranking method, wherein the ranking method selects the plurality of preventive measures based on a minimum distance between a preventive measure and an attack persona.
16. The one or more non-transitory machine readable information storage mediums of claim 15 , wherein the data comprises textual data, binary data and multimedia data, wherein the data is one of, a structured data, an unstructured data and a semi-structured data, and wherein the plurality of external data sources comprising online and offline data sources.
17. The one or more non-transitory machine readable information storage mediums of claim 15 , wherein the plurality of feature engineering techniques comprising a bag of words, a scalarisation, a normalization, a recoding and a feature transformation.
18. The one or more non-transitory machine readable information storage mediums of claim 15 , wherein computing the poisoned data set based on the plurality of feature vectors by using the label flipping comprises:
receiving a flip budget, the data, the plurality of feature vectors and the corresponding label vectors, wherein the flip budget is a pre-computed based on a plurality of constraints comprising a number of queries and a type of updation performed by the ML model comprising one of, a batch updation and an online updation;
segmenting the data into a training data and a testing data;
computing a loss function based on the training data, the plurality of feature vectors of the training data and the corresponding label vectors by using an optimization algorithm, wherein the optimization algorithm comprises one of, a gradient based optimization algorithm and a non-gradient based algorithm;
sorting the training data based on the computed loss function;
selecting a plurality of data points from the sorted training data based on the flip budget; and
flipping one or more label vectors of the plurality of selected data points to generate the poisoned data set.
19. The one or more non-transitory machine readable information storage mediums of claim 15 , wherein the plurality of threat attributes comprises an attacker's skill, an attack time constraint, a percentage of data accessible to an attacker, a number of attacker queries, a complexity of an attack, a transferability of the attack, an attacker's knowledge about an ML model, an attacker's reachability to the dataset and an attacker's goal.
20. The one or more non-transitory machine readable information storage mediums of claim 15 , wherein the plurality of system constraints comprises a number of queries, a type of updation performed by the ML model, a network latency, a system load, a query processing time, a processing limit of a data poisoning algorithm, a time delay between successive queries, a percentage of total data size and a size of the dataset.