Discovering critical alerts through learning over heterogeneous temporal graphs
A method is provided that includes transforming training data into a neural network based learning model using a set of temporal graphs derived from the training data. The method includes performing model learning on the learning model by automatically adjusting learning model parameters based on the set of the temporal graphs to minimize differences between a predetermined ground-truth ranking list and a learning model output ranking list. The method includes transforming testing data into a neural network based inference model using another set of temporal graphs derived from the testing data. The method includes performing model inference by applying the inference and learning models to test data to extract context features for alerts in the test data and calculate a ranking list for the alerts based on the extracted context features. Top-ranked alerts are identified as critical alerts. Each alert represents an anomaly in the test data.
1. A computer-implemented method, comprising:
transforming, by a processor, training data into a neural network based learning model using a set of temporal graphs derived from the training data;
performing, by the processor, model learning on the learning model by automatically adjusting learning model parameters based on the set of the temporal graphs to minimize differences between a predetermined ground-truth ranking list and a learning model output ranking list;
transforming, by the processor, testing data into a neural network based inference model using another set of temporal graphs derived from the testing data; and
performing, by the processor, model inference by applying the inference model and the learning model to test data to extract context features for alerts in the test data and calculate a ranking list for the alerts in the test data based on the extracted context features, wherein top-ranked ones of the alerts in the ranking list are identified as critical alerts, each of the alerts representing an anomaly in the test data.
2. The computer-implemented method of claim 1 , wherein performing the model learning comprises performing structural and temporal graph propagation over the set of temporal graphs.
3. The computer-implemented method of claim 1 , wherein the training data comprises logs, alerts, and user feedback on alerts.
4. The computer-implemented method of claim 1 , wherein transforming the training data comprises:
transforming the training data into a single temporal graph;
calculating, from the single temporal graph, the set of temporal graphs that form a graph sequence; and
transforming the graph sequence into the learning model.
5. The computer-implemented method of claim 1 , wherein the learning model parameters are automatically adjusted using a back propagation through time technique.
6. The computer-implemented method of claim 1 , wherein the ground-truth ranking list is derived from user feedback on alerts.
7. The computer-implemented method of claim 1 , wherein the set of temporal graphs comprises four types of graph nodes, the four types of graph nodes being a log node type, an alert node type, a server node type, and a software node type.
8. The computer-implemented method of claim 7 , wherein the set of temporal graphs further =comprises edges representing dependencies between the graph nodes and timestamps on the edges representing a time of interaction therebetween.
9. The computer-implemented method of claim 1 , wherein states of nodes in the set of temporal graphs are dynamically determined based on updated hidden vectors of the learning model to increase an expressive power of the learning model.
10. The computer-implemented method of claim 1 , wherein influence between nodes in the set of temporal graphs is dynamically determined based on updated hidden vectors of the learning model to increase an expressive power of the learning model.
11. The computer-implemented method of claim 1 , further comprising performing a curative action to overcome an anomaly, responsive to one or more of the identified critical alerts.
12. The computer-implemented method of claim 1 , wherein transforming the testing data comprises:
transforming the testing data into a single temporal graph;
calculating, from the single temporal graph, the other set of temporal graphs that form a graph sequence; and
transforming the graph sequence into the inference model.
13. A computer program product, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
transforming, by a processor of the computer, training data into a neural network based learning model using a set of temporal graphs derived from the training data;
performing, by the processor, model learning on the learning model by automatically adjusting learning model parameters based on the set of the temporal graphs to minimize differences between a predetermined ground-truth ranking list and a learning model output ranking list;
transforming, by the processor, testing data into a neural network based inference model using another set of temporal graphs derived from the testing data; and
performing, by the processor, model inference by applying the inference model and the learning model to test data to extract context features for alerts in the test data and calculate a ranking list for the alerts in the test data based on the extracted context features, wherein top-ranked ones of the alerts in the ranking list are identified as critical alerts, each of the alerts representing an anomaly in the test data.
14. The computer program product of claim 13 , wherein performing the model learning comprises performing structural and temporal graph propagation over the set of temporal graphs.
15. The computer program product of claim 13 , wherein the training data comprises logs, alerts, and user feedback on alerts.
16. The computer program product of claim 13 , wherein transforming the training data comprises:
transforming the training data into a single temporal graph;
calculating, from the single temporal graph, the set of temporal graphs that form a graph sequence; and
transforming the graph sequence into the learning model.
17. The computer program product of claim 13 , wherein the learning model parameters are automatically adjusted using a back propagation through time technique.
18. The computer program product of claim 13 , wherein the ground-truth ranking list is derived from user feedback on alerts.
19. The computer program product of claim 13 , wherein the set of temporal graphs comprises four types of graph nodes, the four types of graph nodes being a log node type, an alert node type, a server node type, and a software node type.
20. A computer processing system, comprising:
a processor configured to:
transform training data into a neural network based learning model using a set of temporal graphs derived from the training data;
perform model learning on the learning model by automatically adjusting learning model parameters based on the set of the temporal graphs to minimize differences between a predetermined ground-truth ranking list and a learning model output ranking list;
transform testing data into a neural network based inference model using another set of temporal graphs derived from the testing data; and
perform model inference by applying the inference model and the learning model to test data to extract context features for alerts in the test data and calculate a ranking list for the alerts in the test data based on the extracted context features, wherein top-ranked ones of the alerts in the ranking list are identified as critical alerts, each of the alerts representing an anomaly in the test data.