Detecting and mitigating system anomalies using knowledge graphs
Detecting and mitigating anomalous system behavior by providing a machine learning model comprising a knowledge graph depicting system entity relationships, and modeling behavioral correlations among system entities according to historical time-series data, receiving real-time time-series data for the system, detecting an anomalous system behavior in a system locale, according to the real-time time-series data, according to the machine learning model and multivariate sensor metrics, diagnosing the anomalous system behavior according to an upstream portion of the knowledge graph and a statistical behavior model for the system locale, and mitigating the anomalous behavior by deriving a recommended action according to the anomalous behavior and generating a work order to implement the recommended action.
1 . A computer implemented method comprising:
providing a machine learning model comprising a knowledge graph depicting system entity relationships, and modeling behavioral correlations among system entities according to historical time-series data;
receiving real-time time-series data for the system according to monitoring frequencies;
detecting an anomalous system behavior in a system locale, according to the real-time time-series data, the machine learning model, monitoring thresholds and multivariate sensor metrics;
adjusting the monitoring thresholds, and the monitoring frequencies, according to the system locale, the knowledge graph and the anomalous system behavior;
diagnosing the anomalous system behavior according to an upstream portion of the knowledge graph and a statistical behavior model for the system locale;
deriving a recommended action according to the anomalous system behavior; and
generating a work order to implement the recommended action.
2 . The computer implemented method according to claim 1 , wherein providing a machine learning model comprising a knowledge graph depicting system entity relationships, and modeling behavioral correlations among system entities according to historical time-series data comprises:
receiving a representation of physical or logical connectivity of the system and historical time series data recorded by sensors of the system;
deriving from at least one of the knowledge graph and historical time series data, a multi-variate statistical model of system-level behavior anomalies; and
deriving from the multi-variate statistical model an indicative set of conditions on multi-variate aggregations of sensors that indicate anomalous behavior.
3 . The computer implemented method according to claim 2 , further comprising:
receiving a prescription for or computing salient sub-graphs and applying it to the knowledge graph to produce a set of one or more salient sub-graphs;
deriving from at least one of the salient sub-graphs and historical time series data, a second multi-variate statistical model of system-level behavior anomalies; and
deriving from the second multi-variate statistical model an indicative set of conditions on multi-variate aggregations of sensors that indicate anomalous behavior.
4 . The computer implemented method according to claim 2 , wherein deriving from at least one of the knowledge graph and historical time series data, a multi-variate statistical model of system-level behavior anomalies comprises limiting the knowledge graph to a set of nodes defined by a maximum inter-node distance from each other.
5 . The computer implemented method according to claim 2 , wherein deriving a multi-variate statistical model comprises inferring causation from directed edges of the knowledge graph.
6 . The computer implemented method according to claim 2 , wherein deriving a multi-variate statistical model comprises encoding a portion of nodes of the knowledge graph as an intermediate representation.
7 . The computer implemented method according to claim 2 , wherein deriving a multi-variate statistical model comprises inferring a Bayes net from the knowledge graph.
8 . The computer implemented method according to claim 1 , further comprising altering an anomalous behavior threshold for a system node selected from a group consisting of nodes upstream of the system locale, nodes downstream of the system locale, and combinations thereof.
9 . A computer program product comprising one or more computer readable storage media and collectively stored program instructions on the one or more computer readable storage media, the stored program instructions which, when executed, cause one or more processors to:
provide a machine learning model comprising a knowledge graph depicting system entity relationships, and modeling behavioral correlations among system entities according to historical time-series data;
receive real-time time-series data for the system according to monitoring frequencies;
detect an anomalous system behavior in a system locale, according to the real-time time-series data, the machine learning model, monitoring thresholds, and multivariate sensor metrics;
adjusting the monitoring thresholds, and the monitoring frequencies, according to the system locale, knowledge graph and the anomalous system behavior;
diagnose the anomalous system behavior according to an upstream portion of the knowledge graph and a statistical behavior model for the system locale;
derive a recommended action according to the anomalous system behavior; and
generate a work order to implement the recommended action.
10 . The computer program product according to claim 9 , wherein providing a machine learning model comprises program instructions, which, when executed cause the one or more processors to:
receive a representation of physical or logical connectivity of the system and historical time series data recorded by sensors of the system;
derive from at least one of the knowledge graph and historical time series data, a multi-variate statistical model of system-level behavior anomalies; and
derive from the multi-variate statistical model an indicative set of conditions on multi-variate aggregations of sensors that indicate anomalous behavior.
11 . The computer program product according to claim 10 , the stored program instructions further causing the one or more processors to:
receive a prescription for or computing salient sub-graphs and applying it to the knowledge graph to produce a set of one or more salient sub-graphs;
derive from at least one of the salient sub-graphs and historical time series data, a second multi-variate statistical model of system-level behavior anomalies; and
derive from the second multi-variate statistical model an indicative set of conditions on multi-variate aggregations of sensors that indicate anomalous behavior.
12 . The computer program product according to claim 10 , wherein deriving from at least one of the knowledge graph and historical time series data, a multi-variate statistical model of system-level behavior anomalies comprises limiting the knowledge graph to a set of nodes defined by a maximum inter-node distance from each other.
13 . The computer program product according to claim 10 , wherein deriving a multi-variate statistical model comprises inferring causation from directed edges of the knowledge graph.
14 . The computer program product according to claim 10 , wherein deriving a multi-variate statistical model comprises encoding a portion of nodes of the knowledge graph as an intermediate representation.
15 . The computer program product according to claim 10 , wherein deriving a multi-variate statistical model comprises inferring a Bayes net from the knowledge graph.
16 . The computer program product according to claim 9 , the stored program instructions further causing the one or more processors to alter an anomalous behavior threshold for a system node selected from a group consisting of nodes upstream of the system locale, nodes downstream of the system locale, and combinations thereof.
17 . A computer system comprising:
one or more computer processors;
one or more computer readable storage media; and
stored program instructions on the one or more computer readable storage media for execution by the one or more computer processors, the stored program instructions which, when executed, cause the one or more computer processors to:
provide a machine learning model comprising a knowledge graph depicting system entity relationships, and modeling behavioral correlations among system entities according to historical time-series data;
receive real-time time-series data for the system according to monitoring frequencies;
detect an anomalous system behavior in a system locale, according to the real-time time-series data, the machine learning model, monitoring thresholds, and multivariate sensor metrics;
adjusting the monitoring thresholds, and the monitoring frequencies, according to the system locale, knowledge graph and the anomalous system behavior;
diagnose the anomalous system behavior according to an upstream portion of the knowledge graph and a statistical behavior model for the system locale;
derive a recommended action according to the anomalous system behavior; and
generate a work order to implement the recommended action.
18 . The computer system according to claim 17 , wherein providing a machine learning model comprises program instructions, which, when executed cause the one or more processors to:
receive a representation of physical or logical connectivity of the system and historical time series data recorded by sensors of the system;
derive from at least one of the knowledge graph and historical time series data, a multi-variate statistical model of system-level behavior anomalies; and
derive from the multi-variate statistical model an indicative set of conditions on multi-variate aggregations of sensors that indicate anomalous behavior.
19 . The computer system according to claim 18 , the stored program instructions further causing the one or more processors to:
receive a prescription for or computing salient sub-graphs and applying it to the knowledge graph to produce a set of one or more salient sub-graphs;
derive from at least one of the salient sub-graphs and historical time series data, a second multi-variate statistical model of system-level behavior anomalies; and
derive from the second multi-variate statistical model an indicative set of conditions on multi-variate aggregations of sensors that indicate anomalous behavior.
20 . The computer system according to claim 17 , the stored program instructions further causing the one or more processors to alter an anomalous behavior threshold for a system node selected from a group consisting of nodes upstream of the system locale, nodes downstream of the system locale, and combinations thereof.