Semantics-oriented analysis of log message content
View Patent ↗A log message is processed. The log message to be processed is received. One or more portions of the log message to be separately extracted are identified. A value is extracted from each identified portion. Extracting the value includes using an extraction rule. The extraction rule is associated with the identified portion.
1. A system for computer parsing a log message, comprising:
a communication interface configured to receive the log message, wherein the log message includes a computer log message; and
a processor coupled with the communication interface and configured to:
identify one or more portions of the log message to be separately extracted;
extract from each identified portion a value using an extraction rule associated with the identified portion, wherein an identified portion of the log message can only be extracted by a single extraction rule;
resolve a conflict between at least a first extraction rule and a second extraction rule, including by
determining whether a term of the identified portion of the log message has been identified as a keyword by the first extraction rule and also as a value by the second extraction rule, and
rejecting the first extraction rule due to a determination that the term of the identified portion of the log message has been identified as the keyword by the first extraction rule and as the value by the second extraction rule; and
provide to a user a computer alert based on a result of the second extraction rule stored via a computer network to a computer storage of the system;
wherein the first extraction rule and the second extraction rule are computer parsing rules of a computer parser parsing the computer log message of the system.
2. The system of claim 1 , wherein the processor is further configured to determine a message type of the log message.
3. The system of claim 2 , wherein extracting the value includes determining whether the determined message type matches a specified message type of the extraction rule.
4. The system of claim 2 , wherein the type of the log message is determined based at least in part on a source of the log message.
5. The system of claim 2 , wherein the type of the log message is determined based at least in part on a keyword included in the log message.
6. The system of claim 1 , wherein the processor is further configured to infer a keyword associated with one of the identified portions at least in part by using associated content of another previously received log message.
7. The system of claim 1 , wherein identifying the one or more portions of the log message includes determining that the log message includes one or more keywords of one or more extraction rules.
8. The system of claim 1 , wherein identifying the one or more portions of the log message includes determining a priority processing order of the one or more portions of the log message based at least in part on a priority identifier of the extraction rule associated with each of the identified portions.
9. The system of claim 1 , wherein the extraction rule of at least one of the identified portion is a part of a recognizer that includes a plurality of extraction rules.
10. The system of claim 1 , wherein a different extraction rule is associated with each of the identified portions.
11. The system of claim 1 , wherein the value is a portion of the log message that matches a regular expression of the extraction rule.
12. The system of claim 1 , wherein the processor is further configured to combine a located keyword within the log message with an associated value located in the log message in the event it is determined that the keyword and the associated value match a modifier extraction rule and the associated value is a matched keyword of another extraction rule.
13. The system of claim 1 , wherein the processor is further configured to use a second group of one or more extraction rules to extract one or more portions of the log message that have not been already identified to be separately extracted.
14. The system of claim 13 , wherein using the second group of one or more extraction rules includes matching a single regular expression of a disjunction combination of a rule regular expression from each extraction rule of the second group of one or more extraction rules.
15. The system of claim 1 , wherein the value is stored in a database.
16. The system of claim 15 , wherein the value is stored under a column name associated with a keyword of the extraction rule used to extract the value stored in the database.
17. The system of claim 16 , wherein the column name is normalized to standardize semantically-alike column names.
18. A method for computer parsing a log message, comprising:
receiving the log message, wherein the log message includes a computer log message;
identifying one or more portions of the log message to be separately extracted;
using a processor to extract from each identified portion a value using an extraction rule associated with the identified portion, wherein an identified portion of the log message can only be extracted by a single extraction rule;
resolving a conflict between at least a first extraction rule and a second extraction rule, including by
determining whether a term of the identified portion of the log message has been identified as a keyword by the first extraction rule and also as a value by the second extraction rule, and
rejecting the first extraction rule due to a determination that the term of the identified portion of the log message has been identified as the keyword by the first extraction rule and as the value by the second extraction rule; and
providing to a user a computer alert based on a result of the second extraction rule stored via a computer network to a computer storage of the system;
wherein the first extraction rule and the second extraction rule are computer parsing rules of a computer parser parsing the computer log message of the system.
19. The method of claim 18 , further comprising determining a message type of the log message.
20. A computer program product for computer parsing a log message, the computer program product being embodied in a tangible computer readable storage medium and comprising computer instructions for:
receiving the log message, wherein the log message includes a computer log message;
identifying one or more portions of the log message to be separately extracted;
extracting from each identified portion a value using an extraction rule associated with the identified portion, wherein an identified portion of the log message can only be extracted by a single extraction rule;
resolving a conflict between at least a first extraction rule and a second extraction rule, including by
determining whether a term of the identified portion of the log message has been identified as a keyword by the first extraction rule and also as a value by the second extraction rule, and
rejecting the first extraction rule due to a determination that the term of the identified portion of the log message has been identified as the keyword by the first extraction rule and as the value by the second extraction rule; and
providing to a user a computer alert based on a result of the second extraction rule stored via a computer network to a computer storage of the system;
wherein the first extraction rule and the second extraction rule are computer parsing rules of a computer parser parsing the computer log message of the system.