Filtering redundant events based on a statistical correlation between events

1. A method of filtering fault events generated in a network environment, comprising:

collecting event history data over a period of time, the event history data comprising a record of each fault event generated by a device in the network environment during the period of time;

dividing the event history data into discrete timeframes of a specified duration, wherein fault events occurring in each timeframe are considered to have occurred substantially simultaneously;

determining which fault events occurred in each discrete timeframe;

constructing a simultaneous event matrix representing a count of discrete timeframes within which a first event and a second event occur substantially simultaneously for each distinct pair of fault events from the event history data;

for each distinct pair of fault events, computing a conditional probability of the second event of the pair occurring given that the first event of the pair has occurred from the simultaneous event matrix;

identifying a second event having the conditional probability exceeding a threshold probability as a redundant fault event; and

removing the redundant fault event from an event stream generated by the device in the network environment.

2. The method of claim 1 , further comprising;

determining one or more event expressions comprising Boolean operations between a plurality of fault events;

computing a statistical correlation between each distinct pair of the one or more event expressions and the fault events from the event history data; and

identifying redundant fault events based on the statistical correlation between each distinct pair of the one or more event expressions and the fault events from the event history data.

3. The method of claim 1 , wherein the conditional probability between each distinct pair of fault events is computed only for distinct pairs of fault events occurring in a particular segment of the network environment.

4. The method of claim 1 , wherein the record of each fault event in the event history data comprises an event type identifier and a timestamp indicating when each fault event occurred.

5. The method of claim 1 , wherein the network environment is an Internet Protocol (IP) network.

6. A system for filtering fault events generated in a network environment, comprising:

a processor; and

a memory coupled to the processor and containing

a data collection module operative to collect event history data over a period of time, the event history data comprising a record of each fault event generated by a device in the network environment during the period of time,

a statistical modeling module operative to:

divide the event history data into discrete timeframes of a specified duration, wherein fault events occurring in each timeframe are considered to have occurred substantially simultaneously,

determine which fault events occurred in each discrete timeframe,

construct a simultaneous event matrix representing a count of discrete timeframes within which a first event and a second event occur substantially simultaneously for each distinct pair of fault events from the event history data,

for each distinct pair of fault events, compute a conditional probability of the second event of the pair occurring given that the first event of the pair has occurred from the simultaneous event matrix, and

identify a second event having the conditional probability exceeding a threshold probability as a redundant fault event, and

a statistical correlation filter module operative to remove the redundant fault event from an event stream generated by the device in the network environment.

7. The system of claim 6 , wherein the statistical modeling module is further operative to:

accept one or more event expressions comprising Boolean operations between a plurality of fault events;

compute a statistical correlation between each distinct pair of the one or more event expressions and the fault events from the event history data; and

identify redundant fault events based on the statistical correlation between each distinct pair of the one or more event expressions and the fault events from the event history data.

8. The system of claim 6 , wherein the conditional probability between each distinct pair of fault events is computed only for fault events occurring in a particular segment of the network environment.

9. The system of claim 6 , wherein the record of each fault event in the event history data comprises an event type identifier and a timestamp indicating when each fault event occurred.

10. The system of claim 6 , wherein the network environment is an Internet Protocol (IP) network.

11. A computer readable storage medium having computer executable instructions stored thereon that, when executed by a computer, cause the computer to:

collect event history data comprising a record of each of a plurality of fault events generated by devices on a network during a period of time;

divide the event history data into discrete timeframes of a specified duration;

determine which of the plurality of fault events occurred in each discrete timeframe;

construct a simultaneous event matrix representing a count of discrete timeframes within which a first event and a second event of each distinct pair of the plurality fault events from the event history data occur;

for each distinct pair of the plurality fault events, calculate a conditional probability of the second event of the pair occurring given that the a first event has occurred from the simultaneous event matrix;

receive a threshold probability value; and

generate a list of redundant fault events from each distinct pair of the plurality of fault events having a conditional probability exceeding the threshold probability value.

12. The computer readable storage medium of claim 11 , wherein the list of redundant fault events is utilized to filter redundant fault events from an event stream generated by devices on the network.

13. The computer readable storage medium of claim 11 , having further computer executable instructions stored thereon that, when executed by a computer, cause the computer to:

receive one or more event expressions comprising Boolean operations between fault events;

for each distinct pair of the one or more event expressions and the plurality of fault events from the event history data, calculate a conditional probability of the fault event occurring substantially simultaneously with a condition defined by the event expression given that the condition defined by the event expression has occurred; and

generate a list of redundant fault events from each distinct pair of the one or more event expressions and the plurality of fault events from the event history data having a conditional probability exceeding the threshold probability.

14. The computer readable storage medium of claim 11 , wherein the conditional probability of a second event of the pair occurring substantially simultaneously with a first event of the pair are calculated only for distinct pairs of fault events occurring in a particular segment of the network.

15. The computer readable storage medium of claim 11 , wherein the network is an Internet Protocol (IP) network.