Proactive anomaly detection
Embodiments of the present invention provide computer implemented methods, computer program products and computer systems. For example, embodiments of the present invention can, in response to receiving a request, collect trace data and specifications for a sequence of requests for normal behavior of a microservice application. Embodiments of the present invention can then generate request contextual features from the collected trace data and specification. Embodiments of the present invention can then train a neural network model based on the generated contextual features; and predicting anomalous behavior of the microservice application using the trained neural network model.
1 . A computer-implemented method comprising:
in response to receiving a request, collecting trace data and specifications for a sequence of requests from an application comprising multiple microservices, wherein the collected trace data and specifications comprise respective trace data and specifications collected at each microservice instance of the multiple microservices;
identifying inter-request factors and intra-request factors from the collected trace data and specifications;
generating request contextual features comprising spatial and temporal dependences from the inter-request factors, the intra-request factors, and the collected trace data and specifications;
building and training a recurrent neural network model using the generated contextual features as inputs;
predicting anomalous behavior of a future request of the application based on spatial and temporal dependencies using the trained recurrent neural network model, wherein the anomalous behavior comprises a service level agreement violation, and wherein the predicting further comprises predicting a region of impacted users and an impact on a subset of requests, wherein the impact specifies a type of request that will fail;
generating, in response to prediction of the anomalous behavior, a proactive alert which comprises the region, the impact, and a reason for the prediction of the anomalous behavior; and
providing a system simulation for the predicted anomalous behavior of the future request;
wherein results of the system simulation comprise details of an end-to-end execution flow at each microservice instance of a microservice path of the future request comprising central processing unit usage, memory usage, disk usage, and network usage of the microservice application.
2 . The computer-implemented method of claim 1 , further comprising: generating visualizations associated with the predicted anomalous behavior.
3 . The computer-implemented method of claim 1 , further comprising: generating a root cause report for the predicted anomalous behavior.
4 . The computer-implemented method of claim 1 , wherein the trace data provides a hierarchal data structure that separates logs into individual requests.
5 . The computer-implemented method of claim 1 , wherein the request contextual features comprises:
a data structure that includes three level of information of a request: request-specification, microservice-path and function-path.
6 . The computer-implemented method of claim 1 , further comprising:
predicting a component of one or more services within a cloud infrastructure is responsible for the service level agreement violation using contextual data that includes traces that separate logs into individual requests.
7 . The computer-implemented method of claim 6 , further comprising:
identifying a respective service that caused a specific request of a group of received requests to fail.
8 . A computer program product comprising:
one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising;
program instructions to, in response to receiving a request, collect trace data and specifications for a sequence of requests from an application comprising multiple microservices, wherein the collected trace data and specifications comprise respective trace data and specifications collected at each microservice instance of the multiple microservices;
program instructions to identify inter-request factors and intra-request factors from the collected trace data and specifications;
program instructions to generate request contextual features comprising spatial and temporal dependences from the inter-request factors, the intra-request factors, and the collected trace data and specifications;
program instructions to build and train a recurrent neural network model using the generated contextual features as inputs;
program instructions to predict anomalous behavior of a future request of the application based on spatial and temporal dependencies using the trained recurrent neural network model, wherein the anomalous behavior comprises a service level agreement violation, and wherein the predicting further comprises predicting a region of impacted users and an impact on a subset of requests, wherein the impact specifies a type of request that will fail;
program instructions to generate, in response to prediction of the anomalous behavior, a proactive alert which comprises the region, the impact, and a reason for the prediction of the anomalous behavior; and
program instructions to provide a system simulation for the predicted anomalous behavior of the future request; wherein results of the system simulation comprise details of an end-to-end execution flow at each microservice instance of a microservice path of the future request comprising central processing unit usage, memory usage, disk usage, and network usage of the microservice application.
9 . The computer program product of claim 8 , wherein the program instructions stored on the one or more computer readable storage media further comprise;
program instructions to generate visualizations associated with the predicted anomalous behavior.
10 . The computer program product of claim 8 , wherein the program instructions stored on the one or more computer readable storage media further comprise:
program instructions to generate a root cause report for the predicted anomalous behavior.
11 . The computer program product of claim 8 , wherein the trace data provides a hierarchal data structure that separates logs into individual requests.
12 . The computer program product of claim 8 , wherein the request contextual features comprise:
a data structure that includes three level of information of a request: request-specification, microservice-path and function-path.
13 . A computer system comprising:
one or more computer processors;
one or more computer readable storage media; and
program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising:
program instructions to, in response to receiving a request, collect trace data and specifications for a sequence of requests from an application comprising multiple microservices, wherein the collected trace data and specifications comprise respective trace data and specifications collected at each microservice instance of the multiple microservices;
program instructions to identify inter-request factors and intra-request factors from the collected trace data and specifications;
program instructions to generate request contextual features comprising spatial and temporal dependences from the inter-request factors, the intra-request factors, and the collected trace data and specifications;
program instructions to build and train a recurrent neural network model using the generated contextual features as inputs;
program instructions to predict anomalous behavior of a future request of the application based on spatial and temporal dependencies using the trained recurrent neural network model, wherein the anomalous behavior comprises a service level agreement violation, and wherein the predicting further comprises predicting a region of impacted users and an impact on a subset of requests, wherein the impact specifies a type of request that will fail;
program instructions to generate, in response to prediction of the anomalous behavior, a proactive alert which comprises the region, the impact, and a reason for the prediction of the anomalous behavior; and
program instructions to provide a system simulation for the predicted anomalous behavior of the future request; wherein results of the system simulation comprise details of an end-to-end execution flow at each microservice instance of a microservice path of the future request comprising central processing unit usage, memory usage, disk usage, and network usage of the microservice application.
14 . The computer system of claim 13 , wherein the program instructions stored on the one or more computer readable storage media further comprise:
program instructions to generate visualizations associated with the predicted anomalous behavior.
15 . The computer system of claim 13 , wherein the program instructions stored on the one or more computer readable storage media further comprise:
program instructions to generate a root cause report for the predicted anomalous behavior.