Decomposing events from managed infrastructures using graph entropy
Methods are provided for clustering events. Data is received at an extraction engine from managed infrastructure. Events are converted into alerts and the alerts mapped to a matrix M. One or more common steps are determined from the events and clusters of events are produced relating to the alerts and or events.
1. A method for clustering events, comprising:
receiving data at an extraction engine from managed infrastructure that includes managed infrastructure physical hardware which supports the flow and processing of information;
producing events that relate to the physical hardware of the managed infrastructure;
converting the events into words and subsets used to group the events into clusters that relate to alerts and events indicative of failures or errors in the managed infrastructure;
converting the data into the alerts;
mapping on a display computer system the alerts into a matrix M, factoring the matrix M into A and B, where A is inspected and significant dusters are extracted, and B is used to assign a start and end time to each duster;
determining one or more common steps from the events and producing clusters relating to at least one of the alerts and events and producing dusters of events relating to the failure or errors in the managed infrastructure, where membership in a duster indicates a common factor of the events that is a failure or an actionable problem in the physical hardware of the managed infrastructure directed to supporting the flow and processing of information;
using a source address for each event to assign a graph coordinate to the event with a subset of attributes being extracted for each event and turning that into a vector;
in response to production of the clusters making one or more physical changes in the managed infrastructure hardware to modify a component in the managed infrastructure physical hardware indicated with a failure or error.
2. The method of claim 1 , further comprising:
using the graph coordinates to cluster the events.
3. The method of claim 2 , further comprising:
determining hop and a proximity of the source of the event.
4. The method of claim 3 , further comprising:
calculating one or more distances between the vectors.
5. The method of claim 4 , further comprising:
bringing together events whose vectors are optimally close.
6. The method of claim 1 , further comprising:
collecting alerts into time buckets.
7. The method of claim 1 , further comprising:
breaking messages into subsets of messages that relate to failures or errors in the managed infrastructure.
8. The method of claim 1 , further comprising, normalizing the words and subtexts.
9. The method of claim 8 , further comprising:
mapping the works and subtexts that are normalized to a common, 0.0 and a non-common, 1.0.
10. The method of claim 1 , further comprising:
normalizing entropy for events.
11. The method of claim 1 , further comprising:
assigning entropy to the alerts.
12. The method of claim 11 , further comprising:
running alerts in parallel with activities of the extraction engine.
13. The method of claim 1 , wherein membership in a cluster indicates a common underlying failure in the infrastructure.
14. The method of claim 1 , further comprising: eliminating duplicate copies of repeating data.
15. The method of claim 14 , further comprising: reducing a number of bytes in network data transfers that need to be sent.
16. The method of claim 1 , further comprising:
determining sigalizer common steps to ascertain how many clusters to extract from events.
17. The method of claim 1 , further comprising:
using entropy and a severity of an event for clustering.