MANAGING DATASETS PRODUCED BY ALERT-TRIGGERING SEARCH QUERIES
Systems and methods for managing datasets produced by alert-triggering search queries in data aggregation and analysis systems. An example method may comprise: executing, by one or more processing devices, a search query on a portion of searchable data associated with a time window to produce a dataset comprising one or more results; responsive to determining that at least a portion of the dataset satisfies a triggering condition defining an alert associated with the search query, generating an instance of the alert; associating, by a memory data structure, the instance of the alert with an identifier of the search query and a time parameter specifying the time window; receiving, from a client computing device, a request for the portion of the dataset; and responsive to determining that the portion of the dataset is not stored in the memory in a manner associating it with the instance of the alert, reproducing the portion of the dataset by re-executing the search query in view of the time parameter.
1 . A method, comprising:
executing, by one or more processing devices, a search query on a portion of searchable data associated with a time window to produce a dataset comprising one or more results;
responsive to determining that at least a portion of the dataset satisfies a triggering condition defining an alert associated with the search query, generating an instance of the alert;
associating, by a memory data structure, the instance of the alert with an identifier of the search query and a time parameter specifying the time window;
receiving, from a client computing device, a request for the portion of the dataset; and
responsive to determining that the portion of the dataset is not stored in the memory in a manner associating it with the instance of the alert, reproducing the portion of the dataset by re-executing the search query in view of the time parameter.
2 . The method of claim 1 , further comprising:
storing, in a memory associated with the computer system, the portion of the dataset and an association of the stored portion of the dataset with the instance of the alert.
3 . The method of claim 1 , further comprising:
implementing a file retention policy with respect to datasets stored in the memory, wherein the file retention policy requires deleting certain datasets responsive to evaluating corresponding file retention conditions.
4 . The method of claim 1 , further comprising:
transmitting the copy of the portion of the dataset to the client computing device.
5 . The method of claim 1 , further comprising associating the instance of the alert with an identifier of the triggering condition.
6 . The method of claim 1 , wherein the searchable data includes time-stamped events having portions of raw machine data.
7 . The method of claim 1 , further comprising:
transmitting, to the client computing device, a notification of the instance of the alert.
8 . The method of claim 1 , wherein the client computing device includes at least one of: a desktop computing device or a mobile computing device.
9 . The method of claim 1 , wherein executing the search query on the portion of searchable data includes applying a late binding schema to the data, the late binding schema associated with one or more extraction rules defining one or more fields.
10 . The method of claim 1 , wherein the portion of searchable data includes machine data generated by at least one of a server, a database, an application, or a network.
11 . The method of claim 1 , wherein the search query is execute in near real-time.
12 . The method of claim 1 , wherein the search query is executed on a schedule that is associated with the alert.
13 . The method of claim 1 , wherein the search query and triggering condition together evaluate portions of the searchable data falling within a rolling time window.
14 . The method of claim 1 , wherein the triggering condition requires that the portion of the dataset includes at least a predetermined number of results.
15 . The method of claim 1 , wherein the triggering condition comprises a secondary conditional search on the dataset produced by the search query.
16 . The method of claim 1 , further comprising:
preforming at least one action associated with the alert, wherein the action includes: sending an electronic mail message, creating a Really Simple Syndication (RSS) feed, executing a script, or causing visual display of the alert instance.
17 . A computer system comprising:
a memory; and
one or more processing devices, coupled to the memory, to:
execute a search query on a portion of searchable data associated with a time window to produce a dataset comprising one or more results;
responsive to determining that at least a portion of the dataset satisfies a triggering condition defining an alert associated with the search query, generate an instance of the alert;
associate, by a memory data structure, the instance of the alert with an identifier of the search query and a time parameter specifying the time window;
receive, from a client computing device, a request for the portion of the dataset; and
responsive to determining that the portion of the dataset is not stored in the memory in a manner associating it with the instance of the alert, reproduce the portion of the dataset by re-executing the search query in view of the time parameter.
18 . The computer system of claim 17 , wherein the processing devices are further to:
store, in a memory associated with the computer system, the portion of the dataset and an association of the stored portion of the dataset with the instance of the alert.
19 . The computer system of claim 17 , wherein the processing devices are further to:
implement a file retention policy with respect to datasets stored in the memory, wherein the file retention policy requires deleting certain datasets responsive to evaluating corresponding file retention conditions.
20 . The computer system of claim 17 , wherein the processing devices are further to:
transmit the copy of the portion of the dataset to the client computing device.
21 . The computer system of claim 17 , wherein the processing devices are further to:
associate the instance of the alert with an identifier of the triggering condition.
22 . The computer system of claim 17 , wherein the searchable data includes time-stamped events having portions of raw machine data.
23 . The computer system of claim 17 , wherein the processing devices are further to:
transmit, to the client computing device, a notification of the instance of the alert.
24 . The computer system of claim 17 , wherein executing the search query on the portion of searchable data includes applying a late binding schema to the data, the late binding schema associated with one or more extraction rules defining one or more fields.
25 . A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a computer system, cause the computer system to perform operations comprising:
executing a search query on a portion of searchable data associated with a time window to produce a dataset comprising one or more results;
responsive to determining that at least a portion of the dataset satisfies a triggering condition defining an alert associated with the search query, generating an instance of the alert;
associating, by a memory data structure, the instance of the alert with an identifier of the search query and a time parameter specifying the time window;
receiving, from a client computing device, a request for the portion of the dataset; and
responsive to determining that the portion of the dataset is not stored in the memory in a manner associating it with the instance of the alert, reproducing the portion of the dataset by re-executing the search query in view of the time parameter.
26 . The computer-readable non-transitory storage medium of claim 25 , further comprising executable instructions causing the computer system to:
store, in a memory associated with the computer system, the portion of the dataset and an association of the stored portion of the dataset with the instance of the alert.
27 . The computer-readable non-transitory storage medium of claim 25 , further comprising executable instructions causing the computer system to:
implement a file retention policy with respect to datasets stored in the memory, wherein the file retention policy requires deleting certain datasets responsive to evaluating corresponding file retention conditions.
28 . The computer-readable non-transitory storage medium of claim 25 , further comprising executable instructions causing the computer system to:
transmit the copy of the portion of the dataset to the client computing device.
29 . The computer-readable non-transitory storage medium of claim 25 , further comprising executable instructions causing the computer system to:
associate the instance of the alert with an identifier of the triggering condition.
30 . The computer-readable non-transitory storage medium of claim 25 , further comprising executable instructions causing the computer system to:
transmit, to the client computing device, a notification of the instance of the alert.