Method for determining normal sequences of events
An information handling system monitors events of a first time period, forms sequences from the events (first sequences), and determines normal sequences of the events. In one embodiment, it may also form sequences based upon events of a second time period (second sequences), the second time period later than the first time period, match the first sequences against the second sequences, and remove events of the second sequences from the events of the second time period. The information handling systems may then search for anomalous events in the remaining events. In another embodiment, the normal sequences may represent purchases. The information handling systems may compare purchases of a customer to the normal sequences and determine products of possible interest to the customer based upon the comparison.
1. A method comprising:
classifying by an information handling system events into event types;
forming first sequences based upon events of a first time period, wherein the forming first sequences includes:
determining probabilities of pairs of event types from the events of the first time period; and
concatenating pairs of event types into sequences, wherein the probability of each pair exceeds a threshold probability value;
forming second sequences based upon events of a second time period, the second time period later than the first time period;
matching a portion of sequences of the second sequences to sequences of the first sequences, thereby producing matched sequences of the second sequences,
wherein:
a first element of one of the first sequences matches a second element of one of the second sequences if the first element and the second element are of the same event type; and
the matching includes removing a third element from one of the matched sequences of the second sequences when the one of the matched sequences is matched to another of the first sequences and the third element does not match an element of the another of the first sequences, thereby producing reduced matched sequences of the second sequences;
for each element in each of the reduced matched sequences of the second sequences, removing normal sequences of events upon which the element was based from the events of the second time period to reduce a number of instances of the element needed for investigation; and
searching for anomalous events in the events of the second time period after removing the event for the each element, wherein an amount of time to complete the searching for the anomalous events in the events of the second time period is reduced based on the removing of the normal sequences of events.
2. The method of claim 1 , wherein the forming first sequences comprises limiting the first sequences to sequences that exceed a threshold probability.
3. The method of claim 1 , wherein the forming first sequences comprises:
selecting an integer k>2;
examining third sequences of length between 2 and k based upon consecutive events of the first time period; and
for each sequence E of the third sequences:
determining a probability of E; and
including E in the first sequences if the probability of E exceeds a threshold probability.
4. The method of claim 1 , wherein the searching for anomalous events comprises searching by a clustering algorithm.
5. The method of claim 1 , further comprising verifying the first sequences by a subject-matter expert.
6. The method of claim 1 , further comprising verifying the anomalous events by a subject-matter expert.
7. An information handling system comprising:
a monitor to detect events;
a timer to determine a first time period and a second time period later than the first time period;
an anomalous event detector to:
classify the events into event types;
form first sequences based upon events of the first time period, including limiting the first sequences to sequences that exceed a first threshold probability;
select an integer k>2;
examine third sequences of length between 2 and k based upon consecutive events of the first time period; and
for each sequence E of the third sequences:
determine a probability of E; and
include E in the first sequences if the probability of E exceeds a second threshold probability;
form second sequences based upon events of the second time period;
match a portion of the sequences of the second sequences to sequences of the first sequences, thereby producing matched sequences of the second sequences,
wherein:
a first element of one of the first sequences matches a second element of one of the second sequences if the first element and the second element are of the same event type; and
the matching includes removing a third element from one of the matched sequences of the second sequences when the one of the matched sequences is matched to another of the first sequences and the third element does not match an element of the another of the first sequences, thereby producing reduced matched sequences of the second sequences;
for each element in each of the reduced matched sequences of the second sequences, remove normal sequences of events upon which the element was based from the events of the second time period to reduce a number of instances of the element needed for investigation; and
search for anomalous events in the events of the second time period after removing the events, wherein an amount of time to complete the searching for the anomalous events in the events of the second time period is reduced based on the removing of the normal sequences of events; and
an alarm component to warn of a detection of an anomalous event, wherein the anomalous event may be indicative of a security risk.
8. The information handling system of claim 7 , wherein the anomalous event detector is to form first sequences by generating discrete pair-wise Markov chains at the first time period.
9. The method of claim 1 , wherein the forming first sequences further comprises determining a time interval for occurrence of events of sequences of the first sequences.
10. The method of claim 1 , wherein the forming first sequences further comprises determining a time duration for events of sequences of the first sequences.
11. The method of claim 1 , wherein the probability of each pair of event types (ET 1 , ET 2 ) is the conditional probability that an event of event type ET 2 follows an event of event type ET 1 .
12. The method of claim 1 , wherein the searching for anomalous events is the searching for anomalous events indicative of security risks.
13. The method of claim 1 , wherein:
the matching a portion of sequences of the second sequences to sequences of the first sequences, thereby producing matched sequences of the second sequences is producing a match of a first sequence (ET 1 . . . ET n ) to a second sequence (E 1 . . . E n+k ), wherein each element ET i of the first sequence is an event type and each element E i of the second sequence is an element;
the second sequence contains k elements E il through E ik not matched by elements of the first sequence;
removing these k unmatched elements E il through E ik produces a reduced matched second sequence of n elements E jl through E jn ; and
each element E jm of the second sequence is of the element type of the element ET m of the first sequence and thus the element E jm matches the element ET m .
14. The method of claim 1 , wherein:
the matching a portion of sequences of the second sequences to sequences of the first sequences, thereby producing matched sequences of the second sequences is producing a match of a first sequence (ET 1 . . . ET n ) to a second sequence (E 1 . . . E n+k ), wherein each element ET i of the first sequence is an event type and each element E i of the second sequence is an element;
the second sequence contains k elements E il through E ik not matched by elements of the first sequence;
removing these k unmatched elements E il through E ik produces a reduced matched second sequence of n elements E jl through E jn ; and
each element E jm of the second sequence is of the element type of the element ET m of the first sequence and thus the element E jm matches the element ET m .