Inferring sourcetype based on match rates for rule packages
Embodiments of the present invention are directed to validating extraction rules. In embodiments, a set of events for which field extraction is desired is obtained. Thereafter, an extraction rule is applied to the set of events to extract fields of the events. The application of the extraction rule can be monitored to determine that the applied extraction rule is invalid. Based on the applied extraction rule being invalid, a new extraction rule can be generated to apply to the set of events.
1. A computer-implemented method comprising:
obtaining a set of events, each event comprising a time stamp and a portion of raw machine data, wherein the raw machine data is produced by one or more components within an information technology environment and reflects activity within the information technology environment;
identifying a match rate for a rule package, including a set of extraction rules associated with a particular sourcetype, the match rate indicating an extent at which the set of extraction rules of the rule package correspond with the set of events; and
determining that the set of events corresponds with the particular sourcetype based on the match rate for the rule package including the set of extraction rules associated with the particular sourcetype.
2. The computer-implemented method of claim 1 , wherein the particular sourcetype comprises a type of source providing the raw machine data.
3. The computer-implemented method of claim 1 , further comprising using the particular sourcetype to tag the set of events with the sourcetype.
4. The computer-implemented method of claim 1 , further comprising including a sourcetype field in the set of events, the sourcetype field indicating the particular sourcetype.
5. The computer-implemented method of claim 1 , further comprising using the particular sourcetype to verify appropriate utilization of the rule package.
6. The computer-implemented method of claim 1 , wherein the match rate indicates an extent at which at least one extraction rule of the rule package matches with the set of events.
7. The computer-implemented method of claim 1 , wherein the match rate indicates a percentage or proportion of events within the set of events for which at least one extraction rule in the rule package matches.
8. The computer-implemented method of claim 1 , wherein determining that the set of events corresponds with the particular sourcetype based on the match rate for the rule package includes determining that the match rate exceeds a threshold value.
9. The computer-implemented method of claim 1 , wherein determining that the set of events corresponds with the particular sourcetype based on the match rate for the rule package includes determining that the match rate is one of a set of highest match rates associated with a plurality of rule packages.
10. The computer-implemented method of claim 1 , wherein determining that the set of events corresponds with the particular sourcetype based on the match rate for the rule package includes determining that the match rate exceeds a threshold value and that the match rate is one of a set of highest match rates associated with a plurality of rule packages.
11. The computer-implemented method of claim 1 further comprising:
identifying a second match rate for a second rule package, including a second set of extraction rules associated with a second particular sourcetype, the second match rate indicating an extent at which the second set of extraction rules of the second rule package correspond with the set of events, wherein the second match rate is less than the first match rate.
12. One or more non-transitory computer storage media having instructions stored thereon, wherein the instructions, when executed by a computing device, cause the computing device to:
obtain a set of events, each event comprising a time stamp and a portion of raw machine data, wherein the raw machine data is produced by one or more components within an information technology environment and reflects activity within the information technology environment;
identify a match rate for a rule package, including a set of extraction rules associated with a particular sourcetype, the match rate indicating an extent at which the set of extraction rules of the rule package correspond with the set of events; and
determine that the set of events corresponds with the particular sourcetype based on the match rate for the rule package including the set of extraction rules associated with the particular sourcetype.
13. The one or more non-transitory computer storage media of claim 12 , wherein the particular sourcetype comprises a type of source providing the raw machine data.
14. The one or more non-transitory computer storage media of claim 12 further comprising using the particular sourcetype to tag the set of events with the sourcetype.
15. The one or more non-transitory computer storage media of claim 12 , further comprising including a sourcetype field in the set of events, the sourcetype field indicating the particular sourcetype.
16. The one or more non-transitory computer storage media of claim 12 , further comprising using the particular sourcetype to verify appropriate utilization of the rule package.
17. The one or more non-transitory computer storage media of claim 12 , wherein the match rate indicates an extent at which at least one extraction rule of the rule package matches with the set of events.
18. The one or more non-transitory computer storage media of claim 12 , wherein the match rate indicates a percentage or proportion of events within the set of events for which at least one extraction rule in the rule package matches.
19. The one or more non-transitory computer storage media of claim 12 , wherein determining that the set of events corresponds with the particular sourcetype based on the match rate for the rule package includes determining that the match rate exceeds a threshold value.
20. A computing system comprising:
one or more processors; and
a memory coupled with the one or more processors, the memory having instructions stored thereon, wherein the instructions, when executed by the one or more processors, cause the computing system to:
obtain a set of events, each event comprising a time stamp and a portion of raw machine data, wherein the raw machine data is produced by one or more components within an information technology environment and reflects activity within the information technology environment;
identify a match rate for a rule package, including a set of extraction rules associated with a particular sourcetype, the match rate indicating an extent at which the set of extraction rules of the rule package correspond with the set of events; and
determine that the set of events corresponds with the particular sourcetype based on the match rate for the rule package including the set of extraction rules associated with the particular sourcetype.