Genetic programming for dynamic cybersecurity
Techniques for improved cybersecurity are provided. A plurality of feature subsets are identified, each containing a respective subset of features from a plurality of features included in a set of training security logs. The plurality of feature subsets is modified using one or more genetic programming techniques, and each of the plurality of feature subsets is scored using a plurality of threat classifiers, where the plurality of threat classifiers comprise trained machine learning models. A set of feature subsets is selected, from the plurality of feature subsets, based on the scores. A type classifier is trained based on the set of feature subsets, where the type classifier comprises a trained machine learning model.
1 . A method, comprising:
identifying a plurality of feature subsets, each containing a respective subset of features from a plurality of features included in a set of training security logs;
modifying the plurality of feature subsets using one or more genetic programming techniques, wherein the modifying includes at least one selected from the group consisting of: (i) selecting a first feature subset from the plurality of feature subsets at least partially randomly, selecting a first feature included in the first feature subset at least partially randomly, selecting a second feature not included in the first feature subset at least partially randomly, and mutating the first feature subset by replacing the first feature with the second feature, and (ii) selecting a first feature subset from the plurality of feature subsets at least partially randomly, selecting a second feature subset from the plurality of feature subsets at least partially randomly, and mating the first and second feature subsets to generate two additional feature subsets to include in the plurality of feature subsets, comprising swapping at least a portion of the first feature subset with at least a portion of the second feature subset;
scoring each of the plurality of feature subsets using a first plurality of threat classifiers, wherein the plurality of threat classifiers correspond to trained machine learning models, comprising:
generating, for each respective training security log of the set of training security logs, a respective plurality of classifications based on processing the respective training security log using each of the first plurality of threat classifiers; and
generating a plurality of security log groups based on the respective pluralities of classifications, wherein each respective security log group of the plurality of security log groups corresponds to a respective type of threat;
training a second plurality of threat classifiers based on the scores and the plurality of security log groups, the second plurality of threat classifiers comprising a respective threat classifier for each respective security log group of the plurality of security log groups; and
training a type classifier, based on the plurality of security log groups, to route security logs among the second plurality of threat classifiers based on the types of threat.
2 . The method of claim 1 , wherein scoring each of the plurality of feature subsets comprises:
training a respective threat classifier, of the plurality of threat classifiers, based on each respective security log group of the plurality of security log groups, wherein each respective threat classifier learns to determine whether input security logs represent a true threat;
classifying at least a subset of the set of training security logs using the plurality of threat classifiers; and
labeling the subset of the set of training security logs to indicate which feature subset, of the plurality of feature subsets, resulted in a highest prediction accuracy.
3 . The method of claim 2 , wherein a respective score of each respective feature subset of the plurality of feature subsets is determined based at least in part on:
an accuracy of the respective threat classifier trained based on the respective feature subset; and
a number of the set of training security logs that the respective threat classifier accurately classifies.
4 . The method of claim 1 , further comprising:
receiving a new security log including the plurality of features; and
selecting a first feature subset of the set of feature subsets by processing the new security log using the type classifier.
5 . The method of claim 4 , further comprising:
identifying a first threat classifier trained based on the first feature subset; and classifying the new security log using the first threat classifier.
6 . The method of claim 1 , wherein identifying a plurality of feature subsets comprises, for each respective feature subset of the plurality of feature subsets, selecting features from the plurality of features at least partially randomly.
7 . A system, comprising:
one or more computer processors; and
logic encoded in a storage medium, the logic executable by operation of the one or more computer processors to perform an operation comprising:
identifying a plurality of feature subsets, each containing a respective subset of features from a plurality of features included in a set of training security logs;
modifying the plurality of feature subsets using one or more genetic programming techniques, wherein the modifying includes at least one selected from the group consisting of: (i) selecting a first feature subset from the plurality of feature subsets at least partially randomly, selecting a first feature included in the first feature subset at least partially randomly, selecting a second feature not included in the first feature subset at least partially randomly, and mutating the first feature subset by replacing the first feature with the second feature, and (ii) selecting a first feature subset from the plurality of feature subsets at least partially randomly, selecting a second feature subset from the plurality of feature subsets at least partially randomly, and mating the first and second feature subsets to generate two additional feature subsets to include in the plurality of feature subsets, comprising swapping at least a portion of the first feature subset with at least a portion of the second feature subset;
scoring each of the plurality of feature subsets using a first plurality of threat classifiers, wherein the plurality of threat classifiers correspond to trained machine learning models, comprising:
generating, for each respective training security log of the set of training security logs, a respective plurality of classifications based on processing the respective training security log using each of the first plurality of threat classifiers; and
generating a plurality of security log groups based on the respective pluralities of classifications, wherein each respective security log group of the plurality of security log groups corresponds to a respective type of threat;
training a second plurality of threat classifiers based on the scores and the plurality of security log groups, the second plurality of threat classifiers comprising a respective threat classifier for each respective security log group of the plurality of security log groups; and
training a type classifier, based on the plurality of security log groups, to route security logs among the second plurality of threat classifiers based on the types of threat.
8 . The system of claim 7 , wherein scoring each of the plurality of feature subsets comprises:
training a respective threat classifier, of the plurality of threat classifiers, based on each respective security log group of the plurality of security log groups, wherein each respective threat classifier learns to determine whether input security logs represent a true threat;
classifying at least a subset of the set of training security logs using the plurality of threat classifiers; and
labeling the subset of the set of training security logs to indicate which feature subset, of the plurality of feature subsets, resulted in a highest prediction accuracy.
9 . The system of claim 8 , wherein a respective score of each respective feature subset of the plurality of feature subsets is determined based at least in part on:
an accuracy of the respective threat classifier trained based on the respective feature subset; and
a number of the set of training security logs that the respective threat classifier accurately classifies.
10 . The system of claim 7 , wherein the logic is further executable by operation of the one or more computer processors to perform:
receiving a new security log including the plurality of features;
selecting a first feature subset of the set of feature subsets by processing the new security log using the type classifier; and
identifying a first threat classifier trained based on the first feature subset; and classifying the new security log using the first threat classifier.
11 . A computer product comprising logic encoded in a non-transitory medium, the logic executable by operation of one or more computer processors to perform an operation comprising:
identifying a plurality of feature subsets, each containing a respective subset of features from a plurality of features included in a set of training security logs;
modifying the plurality of feature subsets using one or more genetic programming techniques, wherein the modifying includes at least one selected from the group consisting of: (i) selecting a first feature subset from the plurality of feature subsets at least partially randomly, selecting a first feature included in the first feature subset at least partially randomly, selecting a second feature not included in the first feature subset at least partially randomly, and mutating the first feature subset by replacing the first feature with the second feature, and (ii) selecting a first feature subset from the plurality of feature subsets at least partially randomly, selecting a second feature subset from the plurality of feature subsets at least partially randomly, and mating the first and second feature subsets to generate two additional feature subsets to include in the plurality of feature subsets, comprising swapping at least a portion of the first feature subset with at least a portion of the second feature subset;
scoring each of the plurality of feature subsets using a first plurality of threat classifiers, wherein the plurality of threat classifiers correspond to trained machine learning models, comprising:
generating, for each respective training security log of the set of training security logs, a respective plurality of classifications based on processing the respective training security log using each of the first plurality of threat classifiers; and
generating a plurality of security log groups based on the respective pluralities of classifications, wherein each respective security log group of the plurality of security log groups corresponds to a respective type of threat;
training a second plurality of threat classifiers based on the scores and the plurality of security log groups, the second plurality of threat classifiers comprising a respective threat classifier for each respective security log group of the plurality of security log groups; and
training a type classifier, based on the plurality of security log groups, to route security logs among the second plurality of threat classifiers based on the types of threat.
12 . The computer product of claim 11 , wherein scoring each of the plurality of feature subsets comprises:
training a respective threat classifier, of the plurality of threat classifiers, based on each respective security log group of the plurality of security log groups, wherein each respective threat classifier learns to determine whether input security logs represent a true threat;
classifying at least a subset of the set of training security logs using the plurality of threat classifiers; and
labeling the subset of the set of training security logs to indicate which feature subset, of the plurality of feature subsets, resulted in a highest prediction accuracy.
13 . The computer product of claim 12 , wherein a respective score of each respective feature subset of the plurality of feature subsets is determined based at least in part on:
an accuracy of the respective threat classifier trained based on the respective feature subset; and
a number of the set of training security logs that the respective threat classifier accurately classifies.
14 . The computer product of claim 11 , wherein the logic is further executable by operation of the one or more computer processors to perform:
receiving a new security log including the plurality of features;
selecting a first feature subset of the set of feature subsets by processing the new security log using the type classifier; and
identifying a first threat classifier trained based on the first feature subset; and classifying the new security log using the first threat classifier.