Online anomaly detection using pairwise agreement in heterogeneous model ensemble
Techniques are provided for online anomaly detection using pairwise agreement in a heterogeneous model ensemble. An exemplary contextual model agreement network comprises nodes and transition edges between the nodes, where each node corresponds to a machine learning model and the transition edges between corresponding pairwise machine learning models encode a level of historical agreement between the pairwise machine learning models. In response to an availability of new data observations: features present in the data observations are extracted; a subset of the machine learning models is selected from the machine learning models based on the extracted features; the historical agreement between the selected machine learning models is compared with a current agreement of the selected machine learning models; and an anomaly is detected in the data observations based on the comparison. The contextual model agreement network is optionally updated based on new data observations.
1. A method, comprising:
extracting features from historical training data;
training a plurality of machine learning models using the extracted features;
annotating the features that were represented in the historical training data used to train each machine learning model;
obtaining a contextual model agreement network comprising a plurality of nodes and transition edges between the plurality of nodes, wherein each node corresponds to one of the plurality of machine learning models and the transition edges between corresponding pairwise machine learning models encode a level of historical agreement between the corresponding pairwise machine learning models, for a plurality of subsets of features observed in the historical training data, when the subsets of features are present in the historical training data; and
performing the following steps, using at least one processing device, in response to an availability of one or more data observations:
extracting features present in the one or more data observations;
selecting a subset of machine learning models from the plurality of machine learning models based on the extracted features;
comparing the historical agreement between the selected subset of machine learning models with a current agreement of the selected subset of machine learning models based on predictions by the selected subset of machine learning models; and
detecting an anomaly in the one or more data observations based on the comparison.
2. The method of claim 1 , wherein the transition edges in the contextual model agreement network further encode a level of confidence of the historical agreement between the corresponding pairwise machine learning models, based on a number of samples in the historical training data configuring the subsets of features shared by said machine learning models.
3. The method of claim 2 , further comprising updating the level of confidence of the historical agreement between the corresponding pairwise machine learning models in response to a processing of the one or more data observations.
4. The method of claim 1 , further comprising generating the contextual model agreement network by using a predefined pairwise agreement function that computes the historical agreement between the corresponding pairwise machine learning models.
5. The method of claim 1 , wherein the selecting further comprises obtaining a subgraph of the contextual model agreement network containing transition edges with a substantially same set of features as the extracted features and identifying the machine learning models associated with the nodes in the subgraph.
6. The method of claim 1 , wherein the detecting the anomaly in the one or more data observations further comprises evaluating one or more of whether pairwise machine learning models that historically agree or disagree based on the one or more data observations and whether pairwise machine learning models that historically disagree now agree based on the one or more data observations.
7. The method of claim 6 , wherein said evaluating additionally considers the level of confidence in the historical agreement between the corresponding pairwise machine learning models.
8. The method of claim 1 , further comprising updating the contextual model agreement network to distinguish between anomalies and a new mode of operation.
9. The method of claim 8 , wherein each level of historical agreement in the contextual model agreement network is adapted when a new observation is passed through one or more of the machine learning models using one or more of an exponential smoothing technique and an anomaly-weighted exponential smoothing technique.
10. A system, comprising:
a memory; and
at least one processing device, coupled to the memory, operative to implement the following steps:
extracting features from historical training data;
training a plurality of machine learning models using the extracted features;
annotating the features that were represented in the historical training data used to train each machine learning model;
obtaining a contextual model agreement network comprising a plurality of nodes and transition edges between the plurality of nodes, wherein each node corresponds to one of the plurality of machine learning models and the transition edges between corresponding pairwise machine learning models encode a level of historical agreement between the corresponding pairwise machine learning models, for a plurality of subsets of features observed in the historical training data, when the subsets of features are present in the historical training data; and
performing the following steps, using at least one processing device, in response to an availability of one or more data observations:
extracting features present in the one or more data observations;
selecting a subset of machine learning models from the plurality of machine learning models based on the extracted features;
comparing the historical agreement between the selected subset of machine learning models with a current agreement of the selected subset of machine learning models based on predictions by the selected subset of machine learning models; and
detecting an anomaly in the one or more data observations based on the comparison.
11. The system of claim 10 , wherein the transition edges in the contextual model agreement network further encode a level of confidence of the historical agreement between the corresponding pairwise machine learning models, based on a number of samples in the historical training data configuring the subsets of features shared by said machine learning models.
12. The system of claim 10 , wherein the selecting further comprises obtaining a subgraph of the contextual model agreement network containing transition edges with a substantially same set of features as the extracted features and identifying the machine learning models associated with the nodes in the subgraph.
13. The system of claim 10 , wherein the detecting the anomaly in the one or more data observations further comprises evaluating one or more of whether pairwise machine learning models that historically agree or disagree based on the one or more data observations and whether pairwise machine learning models that historically disagree now agree based on the one or more data observations.
14. The system of claim 10 , wherein each level of historical agreement in the contextual model agreement network is adapted when a new observation is passed through one or more of the machine learning models using one or more of an exponential smoothing technique and an anomaly-weighted exponential smoothing technique.
15. The system of claim 10 , further comprising generating the contextual model agreement network by using a predefined pairwise agreement function that computes the historical agreement between the corresponding pairwise machine learning models.
16. The system of claim 10 , further comprising updating the contextual model agreement network to distinguish between anomalies and a new mode of operation.
17. A computer program product, comprising a non-transitory processor-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by at least one processing device perform the following steps:
extracting features from historical training data;
training a plurality of machine learning models using the extracted features;
annotating the features that were represented in the historical training data used to train each machine learning model;
obtaining a contextual model agreement network comprising a plurality of nodes and transition edges between the plurality of nodes, wherein each node corresponds to one of the plurality of machine learning models and the transition edges between corresponding pairwise machine learning models encode a level of historical agreement between the corresponding pairwise machine learning models, for a plurality of subsets of features observed in the historical training data, when the subsets of features are present in the historical training data; and
performing the following steps, using at least one processing device, in response to an availability of one or more data observations:
extracting features present in the one or more data observations;
selecting a subset of machine learning models from the plurality of machine learning models based on the extracted features;
comparing the historical agreement between the selected subset of machine learning models with a current agreement of the selected subset of machine learning models based on predictions by the selected subset of machine learning models; and
detecting an anomaly in the one or more data observations based on the comparison.
18. The computer program product of claim 17 , wherein the transition edges in the contextual model agreement network further encode a level of confidence of the historical agreement between the corresponding pairwise machine learning models, based on a number of samples in the historical training data configuring the subsets of features shared by said machine learning models.
19. The computer program product of claim 17 , wherein the detecting the anomaly in the one or more data observations further comprises evaluating one or more of whether pairwise machine learning models that historically agree or disagree based on the one or more data observations and whether pairwise machine learning models that historically disagree now agree based on the one or more data observations.
20. The computer program product of claim 17 , further comprising generating the contextual model agreement network by using a predefined pairwise agreement function that computes the historical agreement between the corresponding pairwise machine learning models.