Constructing event distributions via a streaming scoring operation
A method, system and computer-usable medium for performing a streaming scoring operation, comprising: receiving a stream of events, the stream of events comprising a plurality of events; ingesting the plurality of events; extracting features from the plurality of events to provide extracted features; and, generating a streaming scoring value based upon the extracted features.
1. A computer-implementable method for performing a streaming scoring operation, comprising:
receiving a stream of events, the stream of events comprising a plurality of events, each of the plurality of events referring to an occurrence of an action performed by an entity;
ingesting the plurality of events into a streaming query framework;
extracting features from the plurality of events to provide extracted features via the streaming query framework;
generating a streaming scoring value based upon the extracted features via the streaming query framework; and,
using the streaming scoring value to identify anomalous, abnormal, unexpected or malicious behavior associated with the entity.
2. The method of claim 1 , further comprising:
performing a statistical distribution operation on the extracted features when generating the streaming scoring value, the statistical-distribution operation determining probability distributions of the extracted features.
3. The method of claim 1 , further comprising:
generating a scoring container, and wherein
when extracting features from the plurality of events using the scoring container to identify any outliers from the plurality of extracted features.
4. The method of claim 3 , wherein:
the scoring container is one of a plurality of scoring containers; and further comprising
merging the plurality of scoring containers by combining scoring containers across a plurality of time intervals.
5. The method of claim 1 , further comprising:
determining whether an event matches a query associated with a particular feature;
applying the query to the stream of events; and
classifying a plurality of ingested events based upon whether the plurality of ingested events match the query.
6. The method of claim 5 , wherein:
the query is included within a set of queries;
the set of queries comprise a set of defined queries, each of the set of defined queries comprising an associated feature; and,
applying the set of queries to the stream of events results in categorization of each event with respect to features associated with a subset of queries for which each event matches.
7. A system comprising:
a processor;
a data bus coupled to the processor; and
a non-transitory, computer-readable storage medium embodying computer program code, the non-transitory, computer-readable storage medium being coupled to the data bus, the computer program code interacting with a plurality of computer operations and comprising instructions executable by the processor and configured for:
receiving a stream of events, the stream of events comprising a plurality of events, each of the plurality of events referring to an occurrence of an action performed by an entity;
ingesting the plurality of events into a streaming query framework;
extracting features from the plurality of events to provide extracted features via the streaming query framework;
generating a streaming scoring value based upon the extracted features via the streaming query framework; and,
using the streaming scoring value to identify anomalous, abnormal, unexpected or malicious behavior associated with the entity.
8. The system of claim 7 , wherein the instructions are further configured for:
performing a statistical distribution operation on the extracted features when generating the streaming scoring value, the statistical distribution operation determining statistical distributions of the matching features.
9. The system of claim 7 , wherein the instructions are further configured for:
generating a scoring container, and wherein
when extracting features from the plurality of events using the scoring container to identify any outliers from the plurality of events.
10. The system of claim 9 , wherein:
the scoring container is one of a plurality of scoring containers; and further comprising
merging the plurality of scoring containers by combining scoring containers across a plurality of time intervals.
11. The system of claim 7 , wherein:
determining whether an event matches a query associated with a particular feature;
applying the query to the stream of events; and
classifying a plurality of ingested events based upon whether the plurality of ingested events match the query.
12. The system of claim 11 , wherein:
the query is included within a set of queries;
the set of queries comprise a set of defined queries, each of the set of defined queries comprising an associated feature; and,
applying the set of queries to the stream of events results in categorization of each event with respect to features associated with a subset of queries for which each event matches.
13. A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:
receiving a stream of events, the stream of events comprising a plurality of events, each of the plurality of events referring to an occurrence of an action performed by an entity;
ingesting the plurality of events into a streaming query framework;
extracting features from the plurality of events to provide extracted features via the streaming query framework;
generating a streaming scoring value based upon the extracted features via the streaming query framework; and,
using the streaming scoring value to identify anomalous, abnormal, unexpected or malicious behavior associated with the entity.
14. The non-transitory, computer-readable storage medium of claim 13 , wherein the computer executable instructions are further configured for:
performing a statistical distribution operation on the extracted features when generating the streaming scoring value, the statistical distribution operation determining statistical distributions of the extracted features.
15. The non-transitory, computer-readable storage medium of claim 13 , wherein the computer executable instructions are further configured for:
generating a scoring container, and wherein
when extracting features from the plurality of events using the scoring container to identify any outliers from the plurality of events.
16. The non-transitory, computer-readable storage medium of claim 15 , wherein:
the scoring container is one of a plurality of scoring containers; and further comprising
merging the plurality of scoring containers by combining scoring containers across a plurality of time intervals.
17. The non-transitory, computer-readable storage medium of claim 13 , wherein:
determining whether an event matches a query associated with a particular feature;
applying the query to the stream of events; and
classifying a plurality of ingested events based upon whether the plurality of ingested events match the query.
18. The non-transitory, computer-readable storage medium of claim 13 , wherein:
the query is included within a set of queries;
the set of queries comprise a set of defined queries, each of the set of defined queries comprising an associated feature; and,
applying the set of queries to the stream of events results in categorization of each event with respect to features associated with a subset of queries for which each event matches.
19. The non-transitory, computer-readable storage medium of claim 13 , wherein the computer executable instructions are deployable to a client system from a server system at a remote location.
20. The non-transitory, computer-readable storage medium of claim 13 , wherein the computer executable instructions are provided by a service provider to a user on an on-demand basis.