Feature selection for cybersecurity threat disposition

1 . A method for dynamically selecting features and machine learning models for optimal accuracy when determining a threat disposition of a security alert, the method comprising:

training a base machine learning model using a training dataset to generate a trained base machine learning model, wherein the trained base machine learning model is adapted to identify security threats detectable within the training dataset;

determining impacts that features in the training dataset have on the trained base machine learning model when predicting threat dispositions on the security threats using a machine learning feature impact calculator, wherein the machine learning feature impact calculator utilizes a dynamic feature selection technique using data clustering algorithms to select the features in a dynamic way, wherein the determining includes applying at least one interpretability technique selected from a group consisting of: a local interpretable model-agnostic explanation (LIME) technique, and a partial-dependence-plot (PDP) technique;

testing the base machine learning model using the machine learning feature impact calculator by giving variations of the training dataset and analyzing changes to the predicting the threat dispositions on the security threats;

creating subsets of the features, based on threat dispositions, by analyzing the features with their corresponding impacts and placing common features and impacts into the subsets responsive to confidence values associated with classifications exceeding a confidence threshold;

training a plurality of machine learning models for the security threats with the subsets as training data where a corresponding subset is used by one of the machine learning models, wherein the plurality of machine learning models process the corresponding subset of the subsets from the training dataset to produce a high overall predictive accuracy, and the plurality of machine learning models generates a classification output for a different type of security threat;

training a machine learning feature predictor using the training dataset and the subsets of the features to output selected features used to predict the threat dispositions;

receiving a new data instance representing a security event;

selecting features for the new data instance that are dynamically identified as relevant to determining a threat disposition for the new data instance;

selecting a trained machine learning model from the plurality of machine learning models trained based on the selected features, wherein the selecting the trained machine learning model includes identifying, at runtime, the trained machine learning model associated with the dynamically identified features;

generating the classification output of the trained machine learning model; and

determining the threat disposition using the selected trained machine learning model.

2 . The method of claim 1 , wherein the creating the subsets of the features, based on the threat dispositions, by analyzing the features with their corresponding impacts and placing common features and impacts into a first subset of the subsets, further comprises for a first feature in the features, and an impact of the first feature, processing the first feature as an input to the trained base machine learning model to generate the classification output comprising a classification and an associated confidence value;

evaluating the impact that the first feature has on the classification output by analyzing the associated confidence value; and

placing the first feature into a subset of other features that have confidence values that exceed the confidence threshold for the classification.

3 . The method of claim 1 , wherein selecting, the trained machine learning model from the plurality of machine learning models, comprises selecting a subset of two or more trained machine learning models from the plurality of machine learning models for inclusion in an ensemble of trained machine learning models.

4 . The method of claim 1 , wherein the selecting the features for the new data instance, is performed dynamically for the new data instance in a plurality of different new data instances, in response to receiving the new data instance.

5 . The method of claim 1 , wherein the classification output is selected from a group consisting of escalate and close.

6 . A computer program product for dynamically selecting features and machine learning models for optimal accuracy when determining a threat disposition of a security alert, the computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

train a base machine learning model using a training dataset to generate a trained base machine learning model, wherein the trained base machine learning model is adapted to identify security threats detectable within the training dataset;

determine impacts that features in the training dataset have on the trained base machine learning model when predicting threat disposition on the security threats using a machine learning feature impact calculator, wherein the machine learning feature impact calculator utilizes a dynamic feature selection technique using data clustering algorithms to select the features in a dynamic way, wherein the determine includes apply at least one interpretability technique selected from a group consisting of: a local interpretable model-agnostic explanation (LIME) technique, and a partial-dependence-plot (PDP) technique;

test the base machine learning model using the machine learning feature impact calculator by giving variations of the training dataset and analyzing changes to the predicting the threat dispositions on the security threats;

create subsets of the features, based on threat dispositions, by analyzing the features with their corresponding impacts and placing common features and impacts into the subsets responsive to confidence values associated with classifications exceeding a confidence threshold;

train a plurality of machine learning models for the security threats with the subsets as training data where a corresponding subset is used by one of the machine learning models, wherein the plurality of machine learning models process the corresponding subset of the subsets from the training dataset to produce a high overall predictive accuracy, and the plurality of machine learning models generate a classification output for a different type of security threat;

train a machine learning feature predictor using the training dataset and the subsets of the features to output selected features used to predict the threat dispositions;

receive a new data instance representing a security event;

select features for the new data instance that are dynamically identified as relevant to determining a threat disposition for the new data instance;

select a trained machine learning model from the plurality of machine learning models trained based on the selected features, wherein the select the trained machine learning model includes identify, at runtime, the trained machine learning model associated with the dynamically identified features;

generate the classification output of the trained machine learning model; and

determine the threat disposition using the selected trained machine learning model.

7 . The computer program product of claim 6 , wherein the create the subsets of the features, based on the threat dispositions, for a first feature in the features, and an impact of the first feature, causes the computing device to:

process the first feature as an input to the trained base machine learning model to generate the classification output comprising a classification and an associated confidence value;

evaluate the impact that the first feature has on the classification output by analyzing the associated confidence value; and

place the first feature into a subset of other features that have confidence values that exceed the confidence threshold for the classification.

8 . The computer program product of claim 6 , wherein select, the trained machine learning model from the plurality of machine learning models, causes the computing device to select a subset of two or more trained machine learning models from the plurality of machine learning models for inclusion in an ensemble of trained machine learning models.

9 . The computer program product of claim 6 , wherein the select the selected features from the new data instance is performed dynamically for the new data instance in a plurality of different new data instances, in response to receiving the new data instance.

10 . The computer program product of claim 6 , wherein the classification output is selected from a group consisting of escalate and close.

11 . An apparatus for dynamically selecting features and machine learning models for optimal accuracy when determining a threat disposition of a security alert, the apparatus comprising:

a processor; and

a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to:

train a base machine learning model using a training dataset to generate a trained base machine learning model, wherein the trained base machine learning model is adapted to identify security threats detectable within the training dataset;

determine impacts that features in the training dataset have on the trained base machine learning model when predicting threat disposition on the security threats using a machine learning feature impact calculator, wherein the machine learning feature impact calculator utilizes a dynamic feature selection technique using data clustering algorithms to select the features in a dynamic way, wherein the determine includes apply at least one interpretability technique selected from a group consisting of: a local interpretable model-agnostic explanation (LIME) technique, and a partial-dependence-plot (PDP) technique;

test the base machine learning model using the machine learning feature impact calculator by giving variations of the training dataset and analyzing changes to the predicting the threat dispositions on the security threats;

create subsets of the features, based on threat dispositions, by analyzing the features with their corresponding impacts and placing common features and impacts into the subsets responsive to confidence values associated with classifications exceeding a confidence threshold;

train a plurality of machine learning models for the security threats with the subsets as training data where a corresponding subset is used by one of the machine learning models, wherein the plurality of machine learning models process the corresponding subset of the subsets from the training dataset to produce a high overall predictive accuracy, and the plurality of machine learning models generate a classification output for a different type of security threat;

train a machine learning feature predictor using the training dataset and the subsets of the features to output selected features used to predict the threat dispositions;

receive a new data instance representing a security event;

select features for the new data instance that are dynamically identified as relevant to determining a threat disposition for the new data instance;

select a trained machine learning model from the plurality of machine learning models trained based on the selected features, wherein the select the trained machine learning model includes identify, at runtime, the trained machine learning model associated with the dynamically identified features;

generate the classification output of the trained machine learning model; and

determine the threat disposition using the selected trained machine learning model.

12 . The apparatus of claim 11 , wherein the create the subsets of the features, based on the threat dispositions, for a first feature in the features, and an impact of the first feature, causes the processor to:

process the first feature as an input to the trained base machine learning model to generate the classification output comprising a classification and an associated confidence value;

evaluate the impact that the first feature has on the classification output by analyzing the associated confidence value; and

place the first feature into a subset of other features that have confidence values that exceed the confidence threshold for the classification.

13 . The apparatus of claim 11 , wherein the select the trained machine learning model from the plurality of machine learning models causes the processor to select a subset of two or more trained machine learning models from the plurality of machine learning models for inclusion in an ensemble of trained machine learning models.

14 . The apparatus of claim 11 , wherein the classification output is selected from a group consisting of escalate and close.