IP Library Granted Patent US 11,816,321
Granted Patent B1
US 11,816,321 · App. 16/264,517 · Granted Nov 14, 2023

Enhancing extraction rules based on user feedback

Inventors: Li Li (Richmond, CA); Yongxin Su (Richmond, CA); Ting Yuan (Waterloo, CA); Qian Jie Zhong (Vancouver, CA); Yiyun Zhu (Vancouver, CA)
Assignee: Splunk Inc.
G06F3/04847G06F3/0482G06F16/245G06F16/25G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,816,321
App. No.
16/264,517
Granted
Nov 14, 2023
Kind
B1
Abstract

Embodiments of the present invention are directed to enhancing extraction rules utilizing user feedback. In embodiments, a set of extraction rules relevant to an event set are provided for display. Thereafter, a selection of an extraction rule is received and, in response, a set of events matching the selected extraction rule is provided for display. A modification, for example provided by a user, in association with the extraction rule or the set of events is received. Such a modification is then used (e.g., via machine learning) to enhance extraction rules available for performing subsequent data extraction.

Claims (47)

1. A computer-implemented method comprising:

providing for display, via a graphical user interface, a plurality of extraction rules relevant to an event set and corresponding match rates, the plurality of extraction rules identified as relevant to the event set by determining a match rate for each extraction rule indicating a number of events of in the event set that match the extraction rule as compared to a total number of events in the event set, wherein each event in the event set is associated with a timestamp and includes a portion of raw machine data that reflects activity in an information technology environment and that is produced by a component of that information technology environment, and wherein each extraction rule, of the plurality of extraction rules, indicates how to extract a subportion of text from the portion of raw machine data in the event to produce a value for a field specified by the extraction rule;

receiving, via the graphical user interface, a selection of an extraction rule of the plurality of extraction rules;

providing for display, via the graphical user interface, a set of events matching the selected extraction rule;

receiving, via the graphical user interface, a modification applied to the displayed extraction rule and a modification applied to a field name associated with the set of events matching the selected extraction rule; and

providing the modifications, provided via the graphical user interface, into a machine learning model to enhance extraction rules available for performing subsequent data extraction, wherein the machine learning model uses the modifications to enhance the extraction rules by reordering extraction rules.

2. The computer-implemented method of claim 1 further comprising:

obtaining the event set; and

anonymizing at least a portion of data in the event set.

3. The computer-implemented method of claim 1 further comprising anonymizing at least a portion of data in the event set by replacing the at least the portion of the data with one or more default values.

4. The computer-implemented method of claim 1 , wherein the plurality of extraction rules identified as relevant is based on the corresponding match rates exceeding a threshold.

5. The computer-implemented method of claim 1 further comprising determining the plurality of extraction rules relevant to the event set by comparing extraction rules to the event set.

6. The computer-implemented method of claim 1 wherein the plurality of extraction rules identified as relevant correspond with high match rates.

7. The computer-implemented method of claim 1 , wherein the set of events matching the selected extraction rule are presented with values extracted via the selected extraction rule being visually emphasized within the set of events.

8. The computer-implemented method of claim 1 , further comprising receiving a modification of a field value.

9. The computer-implemented method of claim 1 , wherein the machine learning model further uses the modifications to enhance the extractions rules by replacing extraction rules or refining extraction rules.

10. The computer-implemented method of claim 1 further comprising using at least a portion of the enhanced extraction rules to perform data extraction.

11. The computer-implemented method of claim 1 further comprising:

identifying context associated with the modification applied to the displayed extraction rule; and

providing the context into the machine learning model to generate the enhanced extraction rules.

12. One or more computer storage media having instructions stored thereon, wherein the instructions, when executed by a computing device, cause the computing device to:

provide for display, via a graphical user interface, a plurality of extraction rules relevant to an event set and corresponding match rates, the plurality of extraction rules identified as relevant to the event set by determining a match rate for each extraction rule indicating a number of events of in the event set that match the extraction rule as compared to a total number of events in the event set, wherein each event in the event set is associated with a timestamp and includes a portion of raw machine data that reflects activity in an information technology environment and that is produced by a component of that information technology environment, and wherein each extraction rule, of the plurality of extraction rules, indicates how to extract a subportion of text from the portion of raw machine data in the event to produce a value for a field specified by the extraction rule;

receive, via the graphical user interface, a selection of an extraction rule of the plurality of extraction rules;

provide for display, via the graphical user interface, a set of events matching the selected extraction rule;

receive, via the graphical user interface, a modification applied to the displayed extraction rule and a modification applied to a field name associated with the set of events matching the selected extraction rule; and

provide the modifications, provided via the graphical user interface, into a machine learning model to enhance extraction rules available for performing subsequent data extraction, wherein the machine learning model uses the modifications to enhance the extraction rules by reordering extraction rules.

13. The computer storage media of claim 12 further causing the computing device to:

obtain the event set; and

anonymize at least a portion of data in the event set.

14. The computer storage media of claim 12 further causing the computing device to anonymize at least a portion of data in the event set by replacing the at least the portion of the data with one or more default values.

15. The computer storage media of claim 12 further causing the computing device to determine the plurality of extraction rules relevant to the event set by comparing extraction rules to the event set.

16. The computer storage media of claim 12 , wherein the plurality of extraction rules identified as relevant correspond with high match rates.

17. The computer storage media of claim 12 , wherein the set of events matching the selected extraction rule are presented with values extracted via the selected extraction rule being visually emphasized within the set of events.

18. A computing system comprising:

one or more processors; and

a memory coupled with the one or more processors, the memory having instructions stored thereon, wherein the instructions, when executed by the one or more processors, cause the computing system to:

provide for display, via a graphical user interface, a plurality of extraction rules relevant to an event set and corresponding match rates, the plurality of extraction rules identified as relevant to the event set by determining a match rate for each extraction rule indicating a number of events of in the event set that match the extraction rule as compared to a total number of events in the event set, wherein each event in the event set is associated with a timestamp and includes a portion of raw machine data that reflects activity in an information technology environment and that is produced by a component of that information technology environment, and wherein each extraction rule, of the plurality of extraction rules, indicates how to extract a subportion of text from the portion of raw machine data in the event to produce a value for a field specified by the extraction rule;

receive, via the graphical user interface, a selection of an extraction rule of the plurality of extraction rules;

provide for display, via the graphical user interface, a set of events matching the selected extraction rule;

receive, via the graphical user interface, a modification applied to the displayed extraction rule and a modification applied to a field name associated with the set of events matching the selected extraction rule; and

provide the modifications, provided via the graphical user interface, into a machine learning model to enhance extraction rules available for performing subsequent data extraction, wherein the machine learning model uses the modifications to enhance the extraction rules by reordering extraction rules.

19. The system of claim 18 , further comprising receiving a modification of a field value.

20. The system of claim 18 , wherein the machine learning model further uses the modifications to enhance the extractions rules by replacing extraction rules or refining extraction rules.

21. The system of claim 18 further comprising using at least a portion of the enhanced extraction rules to perform data extraction.

22. The system of claim 18 further comprising:

identifying context associated with the modification to the displayed extraction rule; and

providing the context into the machine learning model to generate the enhanced extraction rules.

Assignments (4)
CHANGE OF NAME Recorded Jul 22, 2025
From: SPLUNK INC.
To: SPLUNK LLC
Reel/Frame 072170/0599 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2025
From: SPLUNK LLC
To: CISCO TECHNOLOGY, INC.
Reel/Frame 072173/0058 →
CHANGE OF NAME Recorded Jan 6, 2025
From: SPLUNK INC.
To: SPLUNK LLC
Reel/Frame 069825/0782 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 26, 2019
From: LI, LI; SU, YONGXIN; YUAN, TING; ZHONG, QIAN JIE; ZHU, YIYUN
To: SPLUNK INC.
Reel/Frame 048704/0713 →