Methods for decomposing events from managed infrastructures
Methods are provided for clustering events. Data is received at an extraction engine from managed infrastructure. Events are converted into alerts and the alerts mapped to a matrix M. One or more common steps are determined from the events and clusters of events are produced relating to the alerts and or events.
1. A method for clustering events, comprising:
receiving messages at an extraction engine from managed infrastructure that includes managed infrastructure physical hardware that supports the flow and processing of information;
producing events that relate to the managed infrastructure and converting the events into words and subsets used to group the events that relate to failures or errors in the managed infrastructure, including the managed infrastructure physical hardware;
providing a sigalizer engine that includes one or more of an NMF engine, a k-means clustering engine and a topology proximity engine, the sigalizer engine determining one or more common characteristics of events and producing clusters of events relating to the failure or errors in the managed infrastructure, the topology proximity engine using a source address for each event and a graph topology of the managed infrastructure which represents node to node connectivity of the topology proximity engine and assigns a graph coordinate to the event with an optional subset of attributes being extracted for each event and turned into a vector, the topology engine inputting a list of devices and a list a connections between components or nodes in the managed infrastructure;
where membership in a cluster indicates a common factor of the events that is a failure or an actionable problem in the physical hardware managed infrastructure directed to supporting the flow and processing of information; and
in response to production of the clusters making one or more physical changes in the managed infrastructure hardware.
2. The method of claim 1 , further comprising:
using a source address for each event to assign a graph coordinate to the event with an optional subset of attributes being extracted for each event and turning that into a vector.
3. The method of claim 2 , further comprising:
using the graph coordinates to cluster the events.
4. The method of claim 1 , further comprising:
determining hop and a proximity of the source of the event.
5. The method of claim 4 , further comprising:
calculating one or more distances between tile vectors.
6. The method of claim 5 , further comprising:
bringing together events whose vectors are optimally close.
7. The method of claim 1 , further comprising:
factoring the matrix M into A and B, where A is inspected and significant clusters are extracted, and B is used to assign a start and end time to each cluster.
8. The method of claim 1 , further comprising:
collecting alerts into time buckets.
9. The method of claim 1 , further comprising:
breaking messages into subsets of messages that relate to failures or errors in the managed infrastructure.
10. The method of claim 4 , further comprising:
grouping subsets of messages into clusters.
11. The method of claim 10 , further comprising:
extracting text components from the event messages and converting them into word and subtext.
12. The method of claim 11 , further comprising, normalizing the works and subtexts.
13. The method of claim 12 , further comprising:
mapping the works and subtexts that are normalized to a common, 0.0 and a non-common, 1.0.
14. The method of claim 1 , further comprising:
normalizing entropy for events.
15. The method of claim 1 , further comprising:
assigning entropy to the alerts.
16. The method of claim 1 , further comprising:
running alerts in parallel with an activities of the extraction engine.
17. The method of claim 1 , wherein membership in a cluster indicates a common underlying failure in the infrastructure.
18. The method of claim 1 , further comprising: eliminating duplicate copies of repeating data.
19. The method of claim 1 , further comprising:
reducing a number of bytes in network data transfers that need to be sent.
20. The method of claim 1 , further comprising: determining sigalizer common steps to ascertain how many clusters to extract from events.
21. The method of claim 1 , further comprising:
using entropy and a severity of an event for clustering.