Dynamically reweighting distributions of event observations
A system, method, and computer-readable medium are disclosed for performing a distribution of interrelated event features operation. The distribution of interrelated event features includes: receiving a stream of events, the stream of events comprising a plurality of events; extracting features from the plurality of events; constructing a distribution of the features from the plurality of events; analyzing the distribution of the features from the plurality of events; and, dynamically reweighting the distribution of the features to scale a number of events contained within the distribution.
1. A computer-implementable method for constructing a distribution of interrelated event features, comprising:
receiving a stream of events, the stream of events comprising a plurality of events;
extracting features from the plurality of events;
constructing a distribution of the features from the plurality of events, the distribution of the features comprising a probability density function distribution of features over a series of time windows;
analyzing the distribution of the features from the plurality of events;
dynamically reweighting the distribution of the features to scale a number of events contained within the distribution; and,
enriching data associated with each of the plurality of events prior to extracting features from the plurality of events; and wherein
the enriching data comprises at least one of
validating event data associated with at least some of the plurality of events,
disclaiming certain event data associated with at least some of the plurality of events;
deduplicating at least some of the plurality of events;
performing an entity resolution operation on at least some of the plurality of events;
performing an attachment enrichment operation on data associated with at least some of the plurality of events; and,
performing a domain enrichment on at least some of the plurality of events.
2. The method of claim 1 , further comprising:
labeling at least some of the plurality of events prior to extracting features from the plurality of events.
3. The method of claim 1 , wherein:
the extracting features comprises performing transformation operations on certain features associated with an event to generate a smaller set of derived features.
4. The method of claim 1 , further comprising:
processing a query relating to the plurality of events, the processing the query being performed via a streaming query framework.
5. A system comprising:
a processor;
a data bus coupled to the processor; and
a non-transitory, computer-readable storage medium embodying computer program code, the non-transitory, computer-readable storage medium being coupled to the data bus, the computer program code interacting with a plurality of computer operations and comprising instructions executable by the processor and configured for:
receiving a stream of events, the stream of events comprising a plurality of events;
extracting features from the plurality of events;
constructing a distribution of the features from the plurality of events, the distribution of the features comprising a probability density function distribution of features over a series of time windows;
analyzing the distribution of the features from the plurality of events:
dynamically reweighting the distribution of the features to scale a number of events contained within the distribution; and,
enriching data associated with each of the plurality of events prior to extracting features from the plurality of events; and wherein
the enriching data comprises at least one of
validating event data associated with at least some of the plurality of events,
disclaiming certain event data associated with at least some of the plurality of events;
deduplicating at least some of the plurality of events;
performing an entity resolution operation on at least some of the plurality of events;
performing an attachment enrichment operation on data associated with at least some of the plurality of events; and,
performing a domain enrichment on at least some of the plurality of events.
6. The system of claim 5 , wherein the instructions are further configured for:
labeling at least some of the plurality of events prior to extracting features from the plurality of events.
7. The system of claim 5 , wherein:
the extracting features comprises performing transformation operations on certain features associated with an event to generate a smaller set of derived features.
8. The system of claim 5 , wherein the instructions are further configured for:
processing a query relating to the plurality of events, the processing the query being performed via a streaming query framework.
9. A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:
receiving a stream of events, the stream of events comprising a plurality of events;
extracting features from the plurality of events;
constructing a distribution of the features from the plurality of events, the distribution of the features comprising a probability density function distribution of features over a series of time windows;
analyzing the distribution of the features from the plurality of events;
dynamically reweighting the distribution of the features to scale a number of events contained within the distribution; and,
enriching data associated with each of the plurality of events prior to extracting features from the plurality of events; and wherein
the enriching data comprises at least one of
validating event data associated with at least some of the plurality of events,
disclaiming certain event data associated with at least some of the plurality of events;
deduplicating at least some of the plurality of events;
performing an entity resolution operation on at least some of the plurality of events;
performing an attachment enrichment operation on data associated with at least some of the plurality of events; and,
performing a domain enrichment on at least some of the plurality of events.
10. The non-transitory, computer-readable storage medium of claim 9 , wherein the computer executable instructions are further configured for:
labeling at least some of the plurality of events prior to extracting features from the plurality of events.
11. The non-transitory, computer-readable storage medium of claim 9 , wherein:
the extracting features comprises performing transformation operations on certain features associated with an event to generate a smaller set of derived features.
12. The non-transitory, computer-readable storage medium of claim 9 , wherein the computer executable instructions are further configured for:
processing a query relating to the plurality of events, the processing the query being performed via a streaming query framework.
13. The non-transitory, computer-readable storage medium of claim 9 , wherein the computer executable instructions are deployable to a client system from a server system at a remote location.
14. The non-transitory, computer-readable storage medium of claim 9 , wherein the computer executable instructions are provided by a service provider to a user on an on-demand basis.