System for decomposing events and unstructured data
A system texecutes automatic attribute inference and includes: a processor; a memory coupled to the memory; a first engine that executes automatic attribute inference; an extraction engine in communication with a managed infrastructure and the first engine, the extraction engine configured to receive managed infrastructure data; and a signaliser engine that includes one or more of an NMF engine, a k-means clustering engine and a topology proximity engine, the signaliser engine inputting a list of devices and a list a connections between components or nodes in the managed infrastructure, the signaliser engine determining one or more common characteristics and produces one or more dusters of events.
1. A system for clustering events received from a managed infrastructure, comprising:
a processor and a floating point unit, wherein the processor is configured to carry out stored program instructions relative to the floating point unit;
a memory coupled to the processor and the floating point unit executing floating point arithmetic logic instructions by retrieving the instructions from the memory of the system, the floating point unit including one or more of a: floating point adder, multiplier and comparator, wherein one or more input signals are propagated through the floating point unit and a resulting performed operation is a floating point unit output, the floating point unit output including a data word and status information of an event that is a rating of a significance of an event
a first engine is coupled to the processor and memory;
an extraction engine in communication with the managed infrastructure and the first engine, the extraction engine configured to receive managed infrastructure data; and
a signaliser engine that includes one or more of an NMF engine, a k-means clustering engine and a topology proximity engine, the signaliser engine inputting a list of devices and a list a connections between components or nodes in the managed infrastructure, the signaliser engine determining one or more common characteristics and produces one or more clusters of events.
2. The system of claim 1 , wherein key attributes of an occurred event of a message are itemized in a first field.
3. The system of claim 2 , further comprising:
a manager that provides for the occurred-event one or more of: an indication of: what the event was, what generated the event, a source of the event; and a host device or host application that generated the event.
4. The system of claim 2 , wherein a type of the event is determined.
5. The system of claim 4 , wherein the event is from one or more of:
a database; and from an application.
6. The system of claim 5 , wherein a plurality of different attributes describes the event to denote a problem that has occurred.
7. The system of claim 2 , wherein a second field represents a source of a message that has generated the message.
8. The system of claim 7 , wherein a third field includes severity information.
9. The system of claim 1 , wherein each of a row in a plurality of rows represents an unstructured message, wherein the plurality of rows is associated with an occurred event.
10. The system of claim 1 , wherein each event record is a different form of a semi structured message.
11. The system of claim 2 , wherein each of a row in a plurality of rows represents a different type of message, wherein the plurality of rows is associated with an occurred event.
12. The system of claim 1 , wherein arbitrary format files containing unstructured data are converted into a structured format that the rest of the system needs to consume.
13. The system of claim 12 , wherein the structured format provides an event record structure.
14. The system of claim 1 , wherein a structured data of a message is an event record.
15. The system of claim 14 , wherein values are assigned to attributes with the attributes being extracted.
16. The system of claim 14 , wherein each of the message has a plurality of components.
17. The system of claim 1 , wherein tokens represent different attributes that a structured message requires.
18. The system of claim 17 , wherein a token comprises at least one of a single word and a group of words.
19. The system of claim 17 , wherein a source attribute is assigned to an event, and different tokens in a message represent different things.
20. The system of claim 17 , wherein a host is identified and assigned to an event.
21. The system of claim 17 , wherein three groups of characters represent month, day, and time.
22. The system of claim 21 , wherein the three groups can represent when an event was created.
23. The system of claim 17 , wherein the system is configured to identify groups of tokens as part of a same event attribute.
24. The system of claim 17 , wherein different types of event attributes are deduced from multiple tokens of a message.
25. The system of claim 17 , wherein an unstructured message is reviewed, and tokens within that message represent different fields that are required in a value record.
26. The system of claim 25 , wherein one token out of the message can represent source, one token can represent a service, and another token can represent a severity.
27. The system of claim 17 , wherein a message is presented to a user with a first part including date and time.
28. The system of claim 17 , wherein a human readable description can be extracted from a log message.
29. The system of claim 17 , wherein the system decides which components from an unstructured message are presented to a user.
30. The system of claim 27 , wherein a second part of the message is an alert classification of a first section.
31. The system of claim 17 , wherein words and tokens are taken out of a log message and assigned to an attribute in an event along with an alert classification.
32. The system of claim 31 , wherein the log message is looked as a whole and assigned an arbitrary label.
33. The system of claim 32 , wherein the labels are not necessarily tokens that appear in the message itself.
34. The system of claim 17 , wherein classification allows assignment of a label to each of an alert.
35. The system of claim 34 , wherein assignment of a label to each of an alert is done for a plurality of attributes, including a class of an event, and a type of an event.
36. The system of claim 1 , wherein a user can graphically configure a data ingestion process, and create training data for the system.
37. The system of claim 36 , wherein system allows a user to create manual overrides for log messages.