Capturing correlations between activity and non-activity attributes using N-grams
View Patent ↗Identifying correlations between events recorded in a computer system log, the recorded events are generated by a plurality of processes executing on the computer. A system log is partitioned into a plurality of segments, each segment associated with a characteristic found in an event, each segment including one or more events having a same characteristic value. A plurality of attributes of the events in a segment are selected. The attributes selected do not describe an action of the event. One or more distinct n-grams are generated, each distinct n-gram including the selected attributes from successive events within the segment. A distinct n-gram is distinct from all other generated n-grams. A correlation is identified for each first selected attribute of each successive event of an n-gram with all other second selected attributes from each successive event of the n-gram, and the correlations are recorded for each first selected attribute.
1. A method for identifying correlations between events recorded in a system log of a computer, the recorded events generated by a plurality of processes executing on the computer, the method comprising:
partitioning, by the computer, a system log into a plurality of segments, each segment associated with a characteristic found in an event, each segment including one or more events having a same characteristic value;
selecting, by the computer, a plurality of attributes of the one or more events in a segment, wherein the plurality of attributes do not describe an action of the event;
generating, by the computer, one or more distinct n-grams, each distinct n-gram including the selected attributes from successive events within the segment, wherein a distinct n-gram is distinct from all other generated n-grams;
identifying, by the computer, a correlation for each first selected attribute of each of the successive events of an n-gram with all other second selected attributes from each of the successive events of the n-gram;
generating, by the computer, a correlation metric as a function of the number of correlated first selected attributes and the total number of selected attributes of each of the successive events of the n-gram, wherein generating the correlation metric includes:
incrementing, by the computer, a count of n-gram instances in which the first selected attribute of each of the successive events of the n-gram correlates with one of the second selected attributes of each of the successive events of the n-gram; and
dividing, by the computer, the count by a total number of possible correlations between the first selected attributes and the second selected attributes; and
recording, by the computer, the correlations for each first selected attribute.
2. The method according to claim 1 , wherein a correlations is one of: always, never, or sometimes.
3. The method according to claim 1 , wherein a correlation is a numeric value between zero and one representing a proportion of n-gram instances out of a total number of n-gram instances for which the first selected attributes correlate with a second selected attribute.
4. The method according to claim 1 , further comprising:
identifying, by the computer, a correlation for each first selected attribute of each of the successive events of one of the one or more n-grams with all other third selected attributes of each event of a different non-overlapping n-gram.