SIGNIFICANT EVENTS IDENTIFIER FOR OUTLIER ROOT CAUSE INVESTIGATION
Embodiments for identifying significant events for finding a root cause of an anomaly collecting time series data for events for each network device by detecting an anomaly in the time series data comprising an outlier on an edge of the time series data by comparing a predicted value of the event to an actual value of the event using a selected forecasting model; declaring the event to be an anomaly at a particular time if a difference between the predicted value and actual value exceed a defined threshold based on residual values for other devices; analyzing in a combined RNN/LSTM process all events for all devices of the network within a time proximity of the particular time of the anomaly to filter usual events and rank each event relative to the anomaly; and displaying a labeled chart of the time series data showing the anomaly in a graph relative to all the events.
1 . A method of identifying significant events for finding a root cause of an anomaly in a network having a server computer, comprising:
collecting time series data for events for each device of the network;
detecting, in a detector component of the server, an anomaly in the time series data comprising an outlier on an edge of the time series data by comparing a predicted value of the event to an actual value of the event using a selected forecasting model;
declaring the event to be an anomaly at a particular time if a difference between the predicted value and actual value exceed a defined threshold based on residual values for other devices of the network;
analyzing, in an analyzer component of the server, all events for all devices of the network within a defined time proximity of the particular time of the anomaly to filter usual events and rank each event relative to the anomaly; and
displaying to a user, through a graphical user interface of a client computer of the network, a labeled chart of the time series data showing the anomaly in a graphical context relative to all the events.
2 . The method of claim 1 wherein the time series data comprises near real-time data as transaction log information written to a central data store, and wherein the events comprise performance metrics of the device and network transactions to and from the device.
3 . The method of claim 2 wherein the analyzing further comprises:
extracting relevant features from the log information;
assigning a value to each feature of the relevant features; and
counting a number of occurrences for each feature value pair in their relative order.
4 . The method of claim 3 wherein the analyzing comprises a Recurrent Neural Network (RNN) process and Markov chain process taking as input a time series of log events and providing as output a probability of a next event to occur or not occur to enable analysis of the next event as normal or not normal.
5 . The method of claim 4 further comprising:
determining, for each of the RNN process and LSTM process, distances between actual events and predicted events; and
calculating a respective score for each log event of the time series of log events based on the distances to help determine a rarity of the next event.
6 . The method of claim 5 further comprising combining the RNN process and the Markov chain process by assigning respective coefficient weights to each of the distances for the RNN process and the Markov chain process.
7 . The method of claim 6 further comprising receiving user feedback of the respective score for each log event, wherein the coefficient weights are determined based on the user feedback using a simple machine learning model, and wherein the score comprises a numeric ranking within a defined range.
8 . The method of claim 7 further comprising calculating an event score for each event by summing a weighted RNN score for an event with a weighted Markov chain score for the event.
9 . The method of claim 8 further comprising labeling the chart with an indexed label identifying each of the events and the anomaly in a contrasting visual manner.
10 . The method of claim 9 wherein the indexed label comprises an alphanumeric character superimposed proximate the events and anomaly, and wherein the chart comprises an interactive chart wherein each indexed label provides an interface providing to information about each event, the information including description, data source, and time of event.
11 . The method of claim 4 wherein the RNN comprises a long short-term memory (LSTM) RNN network.
12 . The method of claim 2 wherein the log information is collected by one of: an agent process embedded in each device of the network, or automatic status transmitting mechanisms native to each device.
13 . A system of identifying significant events for finding a root cause of an anomaly in a network having a server computer, comprising:
a data collector collecting time series data for events for each device of the network;
a detector component of the server detecting an anomaly in the time series data comprising an outlier on an edge of the time series data by comparing a predicted value of the event to an actual value of the event using a selected forecasting model, and declaring the event to be an anomaly at a particular time if a difference between the predicted value and actual value exceed a defined threshold based on residual values for other devices of the network;
an analyzer component of the server analyzing all events for all devices of the network within a defined time proximity of the particular time of the anomaly to filter usual events and rank each event relative to the anomaly; and
a graphical user interface functionally coupled to a client computer of the network displaying a labeled chart of the time series data showing the anomaly in a graphical context relative to all the events.
14 . The system of claim 13 wherein the time series data comprises near real-time data as transaction log information written to a central data store, and wherein the events comprise performance metrics of the device and network transactions to and from the device.
15 . The system of claim 14 wherein the analyzer comprises a Recurrent Neural Network (RNN) process and Markov chain process taking as input a time series of log events and providing as output a probability of a next event to occur or not occur to enable analysis of the next event as normal or not normal, and further extracts relevant features from the log information, assigns a value to each feature of the relevant features, and counts a number of occurrences for each feature value pair in their relative order.
16 . The system of claim 15 wherein the analyzer further determines, for each of the RNN process and LSTM process, distances between actual events and predicted events, and calculates a respective score for each log event of the time series of log events based on the distances to help determine a rarity of the next event.
17 . The system of claim 16 wherein the analyzer combines the RNN process and the Markov chain process by assigning respective coefficient weights to each of the distances for the RNN process and the Markov chain process, and receives user feedback of the respective score for each log event, wherein the coefficient weights are determined based on the user feedback using a simple machine learning model, and wherein the score comprises a numeric ranking within a defined range, and calculates an event score for each event by summing a weighted RNN score for an event with a weighted Markov chain score for the event.
18 . The system of claim 17 wherein the chart is labeled with an indexed label identifying each of the events and the anomaly in a contrasting visual manner, the indexed label comprising an alphanumeric character superimposed proximate the events and anomaly, and wherein the chart comprises an interactive chart wherein each indexed label provides an interface providing to information about each event, the information including description, data source, and time of event.
19 . The system of claim 18 wherein the RNN comprises a long short-term memory (LSTM) RNN network, and wherein the data collector comprises one of an agent process embedded in each device of the network, or automatic status transmitting mechanisms native to each device.
20 . A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to perform a method of identifying significant events for finding a root cause of an anomaly in a network having a server computer, the method comprising:
collecting time series data for events for each device of the network;
detecting, in a detector component of the server, an anomaly in the time series data comprising an outlier on an edge of the time series data by comparing a predicted value of the event to an actual value of the event using a selected forecasting model;
declaring the event to be an anomaly at a particular time if a difference between the predicted value and actual value exceed a defined threshold based on residual values for other devices of the network;
analyzing, in an analyzer component of the server, all events for all devices of the network within a defined time proximity of the particular time of the anomaly to filter usual events and rank each event relative to the anomaly; and
displaying to a user, through a graphical user interface of a client computer of the network, a labeled chart of the time series data showing the anomaly in a graphical context relative to all the events.