Exploring association rules to aid in the trackability of root causes of abnormal events and in the generation of more precise and concise explanations for anomaly detection techniques
One example method includes evaluating a set of itemsets, based on the evaluating, computing association rules corresponding to the itemsets, filtering the association rules to identify relevant association rules, sorting the relevant association rules according to their respective metrics of support, confidence, lift, and conviction, and the relevant association rules are sorted from best metrics to worst metrics, storing (1) best itemsets of the set of itemsets, and (2) the association rules with the best metrics, as thresholds, mapping the thresholds to the stored association rules and to feature-value ranges, and identifying the stored association rules and the feature-value ranges as root causes of an anomaly, and explanations of the anomaly, respectively.
1 . A method, comprising:
generating, by one or more hardware processors, a set of itemsets from boolean transactions generated from discretized outlier-score ranges computed for features of a time-series dataset by an anomaly detection model
evaluating, by the one or more hardware processors, the set of itemsets;
based on the evaluating, computing association rules corresponding to the itemsets, wherein the itemsets are filtered to include itemsets having at least a predefined risk ratio, support, and cardinality indicative of abnormal events;
filtering the association rules to identify relevant association rules whose consequents include at least one itemset associated with an abnormal event;
sorting the relevant association rules according to their respective metrics including support, confidence, lift, and conviction to identify one or more association rules satisfying anomaly explanation criteria;
storing (1) best itemsets selected based on risk ratio, support, and cardinality, and (2) the association rules satisfying the anomaly explanation criteria, as multi-feature anomaly detection thresholds;
mapping the multi-feature anomaly detection thresholds to the stored association rules and to feature-value ranges by identifying, for each feature, subsequences of the time-series dataset having outlier scores within threshold ranges and determining corresponding feature-value ranges;
identifying the stored association rules as root causes of an anomaly and the corresponding feature-value ranges as explanations of the anomaly; and
detecting the anomaly by evaluating antecedent and consequent conditions of at least one stored association rule and determining that the anomaly is present based on satisfaction of both the antecedent and consequent conditions.
2 . The method as recited in claim 1 , wherein each of the itemsets in the set is associated with a respective value of risk ratio, support, and cardinality.
3 . The method as recited in claim 1 , wherein each of the association rules comprises a respective antecedent, and a respective consequent that is associated with the antecedent.
4 . The method as recited in claim 1 , wherein each of the association rules has a respective support ‘s’ and confidence ‘c’.
5 . The method as recited in claim 1 , wherein the root causes of the anomaly and the explanations of the anomaly are used to identify, and implement, a remedial action to resolve the anomaly.
6 . The method as recited in claim 1 , wherein the anomaly is identified by an anomaly detection machine learning model as a result of analysis, by the anomaly detection machine learning model, of time-series data that comprises a set of features (‘F’) and a set of timestamps (‘T’).
7 . The method as recited in claim 1 , wherein the explanations of the anomaly indicate how and/or why the anomaly was identified as such by an anomaly detection machine learning model.
8 . The method as recited in claim 1 , wherein the itemsets in the set were obtained using a frequent pattern mining process.
9 . The method as recited in claim 1 , wherein the association rules are related to combinations of feature-outlier scores.
10 . The method as recited in claim 1 , wherein the support metric indicates a relative frequency of conjunctions of outlier-score ranges.
11 . A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
generating, by one or more hardware processors, a set of itemsets from boolean transactions generated from discretized outlier-score ranges computed for features of a time-series dataset by an anomaly detection model
evaluating, by the one or more hardware processors, the set of itemsets;
based on the evaluating, computing association rules corresponding to the itemsets, wherein the itemsets are filtered to include itemsets having at least a predefined risk ratio, support, and cardinality indicative of abnormal events;
filtering the association rules to identify relevant association rules whose consequents include at least one itemset associated with an abnormal event;
sorting the relevant association rules according to their respective metrics including support, confidence, lift, and conviction to identify one or more association rules satisfying anomaly explanation criteria;
storing (1) best itemsets selected based on risk ratio, support, and cardinality, and (2) the association rules satisfying the anomaly explanation criteria, as multi-feature anomaly detection thresholds;
mapping the multi-feature anomaly detection thresholds to the stored association rules and to feature-value ranges by identifying, for each feature, subsequences of the time-series dataset having outlier scores within threshold ranges and determining corresponding feature-value ranges;
identifying the stored association rules as root causes of an anomaly and the corresponding feature-value ranges as explanations of the anomaly; and
detecting the anomaly by evaluating antecedent and consequent conditions of at least one stored association rule and determining that the anomaly is present based on satisfaction of both the antecedent and consequent conditions.
12 . The non-transitory storage medium as recited in claim 11 , wherein each of the itemsets in the set is associated with a respective value of risk ratio, support, and cardinality.
13 . The non-transitory storage medium as recited in claim 11 , wherein each of the association rules comprises a respective antecedent, and a respective consequent that is associated with the antecedent.
14 . The non-transitory storage medium as recited in claim 11 , wherein each of the association rules has a respective support ‘s’ and confidence ‘c’.
15 . The non-transitory storage medium as recited in claim 11 , wherein the root causes of the anomaly and the explanations of the anomaly are used to identify, and implement, a remedial action to resolve the anomaly.
16 . The non-transitory storage medium as recited in claim 11 , wherein the anomaly is identified by an anomaly detection machine learning model as a result of analysis, by the anomaly detection machine learning model, of time-series data that comprises a set of features (‘F’) and a set of timestamps (‘T’).
17 . The non-transitory storage medium as recited in claim 11 , wherein the explanations of the anomaly indicate how and/or why the anomaly was identified as such by an anomaly detection machine learning model.
18 . The non-transitory storage medium as recited in claim 11 , wherein the itemsets in the set were obtained using a frequent pattern mining process.
19 . The non-transitory storage medium as recited in claim 11 , wherein the association rules are related to combinations of feature-outlier scores.
20 . The non-transitory storage medium as recited in claim 11 , wherein the support metric indicates a relative frequency of conjunctions of outlier-score ranges.