Identifying event distributions using interrelated events
A method, system and computer-usable medium for identifying probability distributions. The identifying probability distributions includes receiving a stream of events, the stream of events comprising a plurality of events; extracting features from the plurality of events, at least some extracted features corresponding to interrelated events; identifying items of interest based upon the interrelated events; and, generating a distribution value based upon the items of interest.
1. A computer-implementable method for identifying probability distributions, comprising:
receiving a stream of events, the stream of events comprising a plurality of events, each of the plurality of events referring to an occurrence of an action performed by an entity;
extracting features from the plurality of events, at least some extracted features corresponding to interrelated events;
identifying items of interest based upon the interrelated events;
generating a distribution value based upon the items of interest, the distribution value comprising a feature score for the items of interest, the feature score being generated based upon a scoring container update operation, the scoring container update operation using a scoring container, the scoring container comprising a container implemented to provide an approximation of a probability distribution over the values the scoring container contains, based upon samples from the probability distribution, the container comprising a data structure storing a collection of objects in an organized way according to an access rule; and,
performing a security analytics operation, the security analytics operation using the distribution value to identify anomalous, abnormal, unexpected or malicious behavior associated with the entity; and wherein
the scoring container is implemented as one or both of a percentile container or a delta container, the percentile container collecting probability distributions of features extracted from the interrelated events to provide percentile probability distributions, the percentile probability distributions of the features then being used to generate the feature score, the delta container collecting probability distributions of features extracted from the interrelated events to provide delta probability distributions, the delta probability distributions of the features being used to update event data.
2. The method of claim 1 , wherein:
each of the plurality of events correspond to a respective time window;
the items of interest are associated with events from a sequence of respective time windows.
3. The method of claim 2 , wherein:
the distribution value of individual features associated with interrelated events corresponding to a sequence of time windows are combined to provide a staggered time window distribution.
4. The method of claim 2 , wherein:
the respective time windows correspond to discrete periods of time; and,
a distribution is generated for each of discrete period of time, the distribution comprising a distribution value based upon events corresponding to the discrete periods of time, each distribution value being iteratively aggregated.
5. The method of claim 2 , wherein:
the respective time windows comprise a series of sequentially generated time windows, the series of sequentially generated time windows being generated from overlapping periods of time.
6. The method of claim 5 , wherein:
the generating the distribution value comprises performing a continuous processing operation, the continuous processing operation continuously processing events from the sequentially generated time windows.
7. A system comprising:
a processor;
a data bus coupled to the processor; and
a non-transitory, computer-readable storage medium embodying computer program code, the non-transitory, computer-readable storage medium being coupled to the data bus, the computer program code interacting with a plurality of computer operations and comprising instructions executable by the processor and configured for:
receiving a stream of events, the stream of events comprising a plurality of events, each of the plurality of events referring to an occurrence of an action performed by an entity;
extracting features from the plurality of events, at least some extracted features corresponding to interrelated events;
identifying items of interest based upon the interrelated events;
generating a distribution value based upon the items of interest, the distribution value comprising a feature score for the items of interest, the feature score being generated based upon a scoring container update operation, the scoring container update operation using a scoring container, the scoring container comprising a container implemented to provide an approximation of a probability distribution over the values the scoring container contains, based upon samples from the probability distribution, the container comprising a data structure storing a collection of objects in an organized way according to an access rule; and,
performing a security analytics operation, the security analytics operation using the distribution value to identify anomalous, abnormal, unexpected or malicious behavior associated with the entity; and wherein
the scoring container is implemented as one or both of a percentile container or a delta container, the percentile container collecting probability distributions of features extracted from the interrelated events to provide percentile probability distributions, the percentile probability distributions of the features then being used to generate the feature score, the delta container collecting probability distributions of features extracted from the interrelated events to provide delta probability distributions, the delta probability distributions of the features being used to update event data.
8. The system of claim 7 , wherein the instructions are further configured for:
each of the plurality of events correspond to a respective time window;
the items of interest are associated with events from a sequence of respective time windows.
9. The system of claim 8 , wherein:
the distribution value of individual features associated with interrelated events corresponding to a sequence of time windows are combined to provide a staggered time window distribution.
10. The system of claim 8 , wherein:
the respective time windows correspond to discrete periods of time; and,
a distribution is generated for each of discrete period of time, the distribution comprising a distribution value based upon events corresponding to the discrete periods of time, each distribution value being iteratively aggregated.
11. The system of claim 8 , wherein:
the respective time windows comprise a series of sequentially generated time windows, the series of sequentially generated time windows being generated from overlapping periods of time.
12. The system of claim 11 , wherein:
the generating the distribution value comprises performing a continuous processing operation, the continuous processing operation continuously processing events from the sequentially generated time windows.
13. A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:
receiving a stream of events, the stream of events comprising a plurality of events, each of the plurality of events referring to an occurrence of an action performed by an entity;
extracting features from the plurality of events, at least some extracted features corresponding to interrelated events;
identifying items of interest based upon the interrelated events;
generating a distribution value based upon the items of interest, the distribution value comprising a feature score for the items of interest, the feature score being generated based upon a scoring container update operation, the scoring container update operation using a scoring container, the scoring container comprising a container implemented to provide an approximation of a probability distribution over the values the scoring container contains, based upon samples from the probability distribution, the container comprising a data structure storing a collection of objects in an organized way according to an access rule; and,
performing a security analytics operation, the security analytics operation using the distribution value to identify anomalous, abnormal, unexpected or malicious behavior associated with the entity; and wherein
the scoring container is implemented as one or both of a percentile container or a delta contain, the percentile container collecting probability distributions of features extracted from the interrelated events to provide percentile probability distributions, the percentile probability distributions of the features then being used to generate the feature score, the delta container collecting probability distributions of features extracted from the interrelated events to provide delta probability distributions, the delta probability distributions of the features being used to update event data.
14. The non-transitory, computer-readable storage medium of claim 13 , wherein the computer executable instructions are further configured for:
each of the plurality of events correspond to a respective time window;
the items of interest are associated with events from a sequence of respective time windows.
15. The non-transitory, computer-readable storage medium of claim 14 , wherein:
the distribution value of individual features associated with interrelated events corresponding to a sequence of time windows are combined to provide a staggered time window distribution.
16. The non-transitory, computer-readable storage medium of claim 14 , wherein:
the respective time windows correspond to discrete periods of time; and,
a distribution is generated for each of discrete period of time, the distribution comprising a distribution value based upon events corresponding to the discrete periods of time, each distribution value being iteratively aggregated.
17. The non-transitory, computer-readable storage medium of claim 14 , wherein:
the respective time windows comprise a series of sequentially generated time windows, the series of sequentially generated time windows being generated from overlapping periods of time.
18. The non-transitory, computer-readable storage medium of claim 17 , wherein:
the generating the distribution value comprises performing a continuous processing operation, the continuous processing operation continuously processing events from the sequentially generated time windows.
19. The non-transitory, computer-readable storage medium of claim 13 , wherein the computer executable instructions are deployable to a client system from a server system at a remote location.
20. The non-transitory, computer-readable storage medium of claim 13 , wherein the computer executable instructions are provided by a service provider to a user on an on-demand basis.