Alert correlating using sequence model with topology reinforcement systems and methods
Alert correlation helps reduce the number of alerts that IT staff have to act upon. Methods include a computer program product that applies a machine driven deep learning model to effectively correlate alerts caused by a common root cause. The methods of correlation provide the user the context of the root cause for the alerts. Therefore, it helps the user to quickly identify, understand and resolve the problem thereby reducing the mean time to identification and resolution. Alerts caused by the same root cause therefor come together.
1. A method comprising:
training, by a system comprising a hardware processor, a sequence model using time sequences of alerts on a recurrent neural network comprising a first long short-term memory layer;
invoking, by the system, the sequence model based on receipt of a first alert;
generating, by the sequence model, a first time sequence of alerts comprising the first alert;
receiving, by the system, a plurality of alerts from one or more applications operating in a network environment comprising computing devices coupled through one or more networks, wherein the plurality of alerts comprise an alert relating to a failure of a computing resource or a program;
generating a second time sequence of alerts including a subset of the plurality of alerts, the subset of the plurality of alerts excluding at least one alert of the plurality of alerts based on the first time sequence of alerts generated by the sequence model, wherein the generating of the second time sequence of alerts comprises:
determining that the at least one alert of the plurality of alerts is not included in the first time sequence of alerts generated by the sequence model, and
excluding the at least one alert from the second time sequence of alerts based on determining that the at least one alert is not included in the first time sequence of alerts generated by the sequence model;
correlating, by the system, the second time sequence of alerts using topology reinforcement to identify a cluster of related alerts;
attributing, by the system, the cluster of related alerts to a common triggering event associated with the failure of the computing resource or the program; and
resolving the failure of the computing resource or the program based on the cluster of related alerts output by the system, the cluster of related alerts providing insight into a root cause of the related alerts, the insight into the root cause used as part of resolving the failure, wherein the recurrent neural network comprises a second long short-term memory layer connected to the first long short-term memory layer, and a dropout layer for regularization, the dropout layer connected to the second long short-term memory layer.
2. The method of claim 1 , further comprising:
converting a raw alert from an application into a normalized alert, the plurality of alerts comprising the normalized alert.
3. The method of claim 1 , wherein the topology reinforcement correlates the second time sequence of alerts by connecting alerts from resources of the network environment that are connected to one another, the connected alerts being the related alerts.
4. The method of claim 1 , wherein the correlating is further according to similarity reinforcement based on natural language processing of the alerts in the second time sequence of alerts.
5. The method of claim 1 , wherein the at least one alert is unrelated to the subset of the plurality of alerts included in the second time sequence of alerts.
6. The method of claim 1 , further comprising:
receiving, by the sequence model, the first alert at a first timestep;
predicting, by the sequence model, alerts at further timesteps after the first timestep; and
including, by the sequence model, the predicted alerts in the first time sequence of alerts.
7. The method of claim 6 , further comprising:
performing, by the sequence model, alert embedding comprising representing alerts as vectors in a vector space.
8. The method of claim 7 , further comprising:
using, by the sequence model, proximities of the vectors representing the alerts in the vector space to identify the related alerts.
9. The method of claim 1 , wherein the plurality of alerts are not analyzed individually or in alert pairs.
10. A non-transitory computer-readable storage medium storing computer-executable instructions that upon execution cause a computer to:
train a sequence model using time sequences of alerts on a recurrent neural network comprising a first long short-term memory layer;
invoke the sequence model based on receipt of a first alert;
generate, by the sequence model, a first time sequence of alerts comprising the first alert;
receive a plurality of alerts from one or more applications operating in a network environment comprising computing devices coupled through one or more networks, wherein the plurality of alerts comprise an alert relating to a failure of a computing resource or a program;
generate a second time sequence of alerts including a subset of the plurality of alerts, the subset of the plurality of alerts excluding at least one alert of the plurality of alerts based on the first time sequence of alerts generated by the sequence model, wherein the generating of the second time sequence of alerts comprises:
determining that the at least one alert of the plurality of alerts is not included in the first time sequence of alerts generated by the sequence model, and
excluding the at least one alert from the second time sequence of alerts based on determining that the at least one alert is not included in the first time sequence of alerts generated by the sequence model;
correlate the second time sequence of alerts using topology reinforcement to identify a cluster of related alerts;
attribute the cluster of related alerts to a common triggering event associated with the failure of the computing resource or the program; and
initiate, by the computer, a resolution of the failure of the computing resource or the program based on the cluster of related alerts identified by the computer, the cluster of related alerts providing insight into a root cause of the related alerts, the insight into the root cause used as part of the resolution of the failure, wherein the recurrent neural network comprises a second long short-term memory layer connected to the first long short-term memory layer, and a dropout layer for regularization, the dropout layer connected to the second long short-term memory layer.
11. The non-transitory computer-readable storage medium of claim 10 , wherein the computer-executable instructions upon execution cause the computer to:
convert a raw alert from an application into a normalized alert, the plurality of alerts comprising the normalized alert.
12. The non-transitory computer-readable storage medium of claim 10 , wherein the topology reinforcement correlates the second time sequence of alerts by connecting alerts from resources of the network environment that are connected to one another, the connected alerts being the related alerts.
13. The non-transitory computer-readable storage medium of claim 10 , wherein the correlating is further according to similarity reinforcement based on natural language processing of the alerts in the second time sequence of alerts.
14. The non-transitory computer-readable storage medium of claim 10 , wherein the at least one alert is unrelated to the subset of the plurality of alerts included in the second time sequence of alerts.
15. The non-transitory computer-readable storage medium of claim 10 , wherein the computer-executable instructions upon execution cause the computer to:
receive, by the sequence model, the first alert at a first timestep;
predict, by the sequence model, alerts at further timesteps after the first timestep; and
include, by the sequence model, the predicted alerts in the first time sequence of alerts.
16. The non-transitory computer-readable storage medium of claim 10 , wherein the computer-executable instructions upon execution cause the computer to:
perform, by the sequence model, alert embedding comprising representing alerts as vectors in a vector space.
17. The non-transitory computer-readable storage medium of claim 16 , wherein the computer-executable instructions upon execution cause the computer to:
use, by the sequence model, proximities of the vectors representing the alerts in the vector space to identify the related alerts.
18. The non-transitory computer-readable storage medium of claim 10 , wherein the plurality of alerts are not analyzed individually or in alert pairs.
19. A system comprising:
a processor; and
a non-transitory storage medium comprising instructions executable on the processor to:
train a sequence model using time sequences of alerts on a recurrent neural network comprising a first long short-term memory layer;
invoke the sequence model based on receipt of a first alert;
generate, with the sequence model, a first time sequence of alerts comprising the first alert;
receive a plurality of alerts from one or more applications operating in a network environment comprising computing devices coupled through one or more networks, wherein the plurality of alerts comprise an alert relating to a failure of a computing resource or a program;
generate a second time sequence of alerts including a subset of the plurality of alerts, the subset of the plurality of alerts excluding at least one alert of the plurality of alerts based on the first time sequence of alerts generated by the sequence model, wherein the generating of the second time sequence of alerts comprises:
determining that the at least one alert of the plurality of alerts is not included in the first time sequence of alerts generated by the sequence model, and
excluding the at least one alert from the second time sequence of alerts based on determining that the at least one alert is not included in the first time sequence of alerts generated by the sequence model;
correlate the second time sequence of alerts using topology reinforcement to identify a cluster of related alerts;
attribute the cluster of related alerts to a common triggering event associated with the failure of the computing resource or the program; and
initiate, by the system, a resolution of the failure of the computing resource or the program based on the cluster of related alerts identified by the system, the cluster of related alerts providing insight into a root cause of the related alerts, the insight into the root cause used as part of the resolution of the failure, wherein the recurrent neural network comprises a second long short-term memory layer connected to the first long short-term memory layer, and a dropout layer for regularization, the dropout layer connected to the second long short-term memory layer.
20. The system of claim 19 , wherein the instructions are executable on the processor to:
convert a raw alert received from an application into a normalized alert, the plurality of alerts comprising the normalized alert.
21. The system of claim 19 , wherein the topology reinforcement correlates the second time sequence of alerts by connecting alerts from resources of the network environment that are connected to one another, the connected alerts being the related alerts.
22. The system of claim 19 , wherein the instructions are executable on the processor to:
perform alert embedding comprising representing alerts as vectors in a vector space.
23. The system of claim 22 , wherein the instructions are executable on the processor to:
use, by the sequence model, proximities of the vectors representing the alerts in the vector space to identify the related alerts.