Alert rule evaluation for monitoring of late arriving data
A monitoring system is configured to distinguish between two types of alert rules—namely, invariant alert rules and variant alert rules—and to apply a different method of alert rule evaluation to each, wherein each alert rule evaluation method deals with the issue of latent data ingestion in a different way. By tailoring the alert rule evaluation method to the type of alert rule being evaluated, the system can apply an optimized approach for each type of alert rule in terms of achieving a trade-off between alert latency, alert accuracy, and cost of goods sold. In an embodiment, the system utilizes a machine learning model to classify a query associated with an alert rule as either increasing or non-increasing. Then, based on the query classification and a condition associated with the alert rule, the system determines if the alert rule is invariant or variant.
1. A monitoring system, comprising:
one or more processors; and
one or more memory devices that store program code that, when to be executed by the one or more processors, cause the monitoring system to:
receive an alert rule that comprises a query, a condition, and a rule frequency;
determine that the alert rule is invariant; and
in response to determining that the alert rule is invariant:
execute, at the rule frequency, the query against a data store for an evaluation time window and for a fixed number of time windows that precede the evaluation time window;
obtain a query result for the evaluation time window and for each time window of the fixed number of time windows that precede the evaluation time window;
for the evaluation time window and each time window of the fixed number of time windows that precede the evaluation time window:
determine that the query result satisfies the condition; and
generate an alert in response to determining that the query result satisfies the condition.
2. The monitoring system of claim 1 , wherein the alert rule is determined to be invariant by:
classifying the query as one of an increasing query or a non-increasing query; and
determining that the alert rule is invariant based on at least the classifying and the condition.
3. The monitoring system of claim 2 , wherein the query is classified as the one of the increasing query or the non-increasing query by:
obtaining a set of features associated with the query; and
providing the set of features as input to a machine learning model that classifies the query as the one of the increasing query or the non-increasing query based on the set of features.
4. The monitoring system of claim 3 , wherein the machine learning model comprises a Random Forest model.
5. The monitoring system of claim 2 , wherein t the query is classified as the one of the increasing query or the non-increasing query by:
analyzing the query result at different points in time; and
classifying the query as the one of the increasing query or the non-increasing query based at least on the analysis.
6. A method for alert rule evaluation in a monitoring system, comprising:
receiving an alert rule, the alert rule comprising a query, a condition, and a rule frequency;
determining that the alert rule is invariant; and
in response to determining that the alert rule is invariant:
executing, at the rule frequency, the query against a data store for an evaluation time window and for a fixed number of time windows that precede the evaluation time window;
obtaining a query result for the evaluation time window and for each time window of the fixed number of time windows that precede the evaluation time window;
for the evaluation time window and for each time window of the fixed number of time windows that precede the evaluation time window:
determining that the query result satisfies the condition; and
generating an alert in response to determining that the query result satisfies the condition.
7. The method of claim 6 , wherein the alert rule is determined to be invariant by:
classifying the query as one of an increasing query or a non-increasing query; and
determining that the alert rule is invariant based on at least the classifying and the condition.
8. The method of claim 7 , wherein the query is classified as the one of the increasing query or the non-increasing query by:
obtaining a set of features associated with the query; and
providing the set of features as input to a machine learning model that classifies the query as the one of the increasing query or the non-increasing query based on the set of features.
9. The method of claim 8 , wherein the machine learning model comprises a Random Forest model.
10. The method of claim 7 , wherein the query is classified as the one of the increasing query or the non-increasing query by:
analyzing the query result at different points in time; and
classifying the query as the one of the increasing query or the non-increasing query based at least on the analysis.
11. One or more memory devices that store program code that, when executed by one or more processors, cause a monitoring system to:
receive an alert rule comprising a query, a condition, and a rule frequency;
determine that the alert rule is invariant; and
in response to determining that the alert rule is invariant:
execute, at the rule frequency, the query against a data store for an evaluation time window and for a fixed number of time windows that precede the evaluation time window;
obtain a query result for the evaluation time window and for each time window of the fixed number of time windows that precede the evaluation time window;
for the evaluation time window and for each time window of the fixed number of time windows that precede the evaluation time window:
determine that the query result satisfies the condition; and
generate an alert in response to determining that the query result satisfies the condition.
12. The one or more memory devices of claim 11 , wherein the alert rule is determined to be invariant by:
classifying the query as one of an increasing query or a non-increasing query; and
determining that the alert rule is invariant based on at least the classifying and the condition.
13. The one or more memory devices of claim 12 , wherein the query is classified as the one of the increasing query or the non-increasing query by:
obtaining a set of features associated with the query; and
providing the set of features as input to a machine learning model that classifies the query as the one of the increasing query or the non-increasing query based on the set of features.
14. The one or more memory devices of claim 13 , wherein the machine learning model comprises a Random Forest model.
15. The one or more memory devices of claim 12 , wherein the query is classified as the one of the increasing query or the non-increasing query by:
analyzing the query result at different points in time; and
classifying the query as the one of the increasing query or the non-increasing query based at least on the analysis.