Identifying security risks using distributions of characteristic features extracted from a plurality of events
A method, system and computer-usable medium for constructing a distribution of interrelated event features. The constructing a distribution of interrelated event features includes receiving a stream of events, the stream of events comprising a plurality of events; extracting features from the plurality of events; constructing a distribution of the features from the plurality of events; and, analyzing the distribution of the features from the plurality of events.
1. A computer-implementable method for constructing a distribution of event features for identifying security risk factors, comprising:
receiving a stream of events, the stream of events comprising a plurality of events, some of the plurality of events comprising multiple feature types, the multiple feature types comprising non-categorical features and categorical features, the non-categorical features comprising at least one of a raw number feature and a binary feature, the categorical features comprising something other than the raw number feature and the binary feature;
extracting a categorical feature from the plurality of events, wherein
the categorical feature includes a set of categorical feature members, and
the categorical feature members are strings having one or more common characteristics defined by the categorical feature that are extracted from events of the stream of events;
constructing a distribution for the categorical feature based on categorical feature members extracted from the plurality of events, the distribution representing a discrete probability distribution, the discrete probability distribution describing a possibility that a categorical feature member of the set of categorical feature members will occur in an event, the constructing the distribution of categorical features comprising performing a scoring container update operation, the scoring container update operation constructing and maintaining probability distribution corresponding to features associated with an event occurring during a particular interval of time; and,
analyzing the distribution of the categorical feature to identify one or more security risk factors.
2. The method of claim 1 , wherein
the set of categorical feature members comprises a fixed set of categorical feature members.
3. The method of claim 1 , wherein
the set of categorical feature members comprises a variable set of categorical feature members that are generated on the fly from string values included in the extracted categorical feature.
4. The method of claim 1 , wherein construction of the distribution comprises:
constructing a categorical distribution for the categorical feature using occurrences of the categorical feature members of the categorical feature extracted from the plurality of events.
5. The method of claim 1 , wherein all constructing the distribution for occurrences of the members of the categorical feature set comprises:
grouping the members of the categorical feature into a plurality of mutually exclusive subsets; and
constructing the distribution using bins respectively associated with the plurality of mutually exclusive subsets.
6. The method of claim 1 , wherein construction of the distribution comprises:
assigning a probability that a given member of the categorical feature set will occur based on a likelihood that a member of the categorical feature set, other than the given member, will occur.
7. The method of claim 1 , further comprising:
mapping the string values to respective one-hot encoded values.
8. The method of claim 1 , wherein the characteristic feature comprises one or more of:
a temporal characteristic feature of an event; and
a communication protocol characteristic of an event.
9. A system comprising:
a processor;
a data bus coupled to the processor; and
a non-transitory, computer-readable storage medium embodying computer program code, the non-transitory, computer-readable storage medium being coupled to the data bus, the computer program code interacting with a plurality of computer operations and comprising instructions executable by the processor and configured for:
receiving a stream of events, the stream of events comprising a plurality of events, some of the plurality of events comprising multiple feature types, the multiple feature types comprising non-categorical features and categorical features, the non-categorical features comprising at least one of a raw number feature and a binary feature, the categorical features comprising something other than the raw number feature and the binary feature;
extracting a categorical feature from the plurality of events, wherein
the categorical feature includes a set of categorical feature members, and
the categorical feature members are strings having one or more common characteristics defined by the categorical feature that are extracted from events of the stream of events;
constructing a distribution for the categorical feature based on categorical feature members extracted from the plurality of events, the distribution representing a discrete probability distribution, the discrete probability distribution describing a possibility that a categorical feature member of the set of categorical feature members will occur in an event; and,
analyzing the distribution of the categorical feature to identify one or more security risk factors, the constructing the distribution of categorical features comprising performing a scoring container update operation, the scoring container update operation constructing and maintaining probability distribution corresponding to features associated with an event occurring during a particular interval of time.
10. The system of claim 9 , wherein
the set of categorical feature members comprises a fixed set of categorical feature members.
11. The system of claim 9 , wherein
the set of categorical feature members comprises a variable set of categorical feature members that are generated on the fly from string values included in the extracted categorical feature.
12. The system of claim 9 , wherein construction of the distribution comprises:
constructing a categorical distribution for the categorical feature using occurrences of the categorical feature members of the categorical feature extracted from the plurality of events.
13. The system of claim 9 , wherein all constructing the distribution for occurrences of the members of the categorical feature set comprises:
grouping the members of the categorical feature into a plurality of mutually exclusive subsets; and
constructing the distribution using bins respectively associated with the plurality of mutually exclusive subsets.
14. The system of claim 9 , wherein construction of the distribution comprises:
assigning a probability that a given member of the categorical feature set will occur based on a likelihood that a member of the categorical feature set, other than the given member, will occur.
15. The system of claim 9 , further comprising:
mapping the string values to respective one-hot encoded values.
16. The system of claim 9 , wherein the characteristic feature comprises one or more of:
a temporal characteristic feature of an event; and
a communication protocol characteristic of an event.
17. A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:
receiving a stream of events, the stream of events comprising a plurality of events, some of the plurality of events comprising multiple feature types, the multiple feature types comprising non-categorical features and categorical features, the non-categorical features comprising at least one of a raw number feature and a binary feature, the categorical features comprising something other than the raw number feature and the binary feature;
extracting a categorical feature from the plurality of events, wherein
the categorical feature includes a set of categorical feature members, and
the categorical feature members are strings having one or more common characteristics defined by the categorical feature that are extracted from events of the stream of events;
constructing a distribution for the categorical feature based on categorical feature members extracted from the plurality of events, the distribution representing a discrete probability distribution, the discrete probability distribution describing a possibility that a categorical feature member of the set of categorical feature members will occur in an event, the constructing the distribution of categorical features comprising performing a scoring container update operation, the scoring container update operation constructing and maintaining probability distribution corresponding to features associated with an event occurring during a particular interval of time; and,
analyzing the distribution of the categorical feature to identify one or more security risk factors.
18. The non-transitory, computer-readable storage medium of claim 17 , wherein
the set of categorical feature members comprises a variable set of categorical feature members that are generated on the fly from string values included in the extracted categorical feature.
19. The method of claim 1 , the instructions are further configured for
converting the strings having one or more common characteristics to numeric values;
assigning the numeric values corresponding to the strings to corresponding bins;
updating counters associated with each bin when a numeric value for the corresponding bin is assigned; and
analyzing the distribution of the counters associated with the bins when identifying the one or more security risk factors.
20. The system of claim 9 , the computer executable instructions are further configured for
converting the strings having one or more common characteristics to numeric values;
assigning the numeric values corresponding to the strings to corresponding bins;
updating counters associated with each bin when a numeric value for the corresponding bin is assigned; and
analyzing the distribution of the counters associated with the bins when identifying the one or more security risk factors.
21. The non-transitory, computer-readable storage medium of claim 17 , wherein
converting the strings having one or more common characteristics to numeric values;
assigning the numeric values corresponding to the strings to corresponding bins;
updating counters associated with each bin when a numeric value for the corresponding bin is assigned; and
analyzing the distribution of the counters associated with the bins when identifying the one or more security risk factors.