System and method for anomaly detection and root cause automation using shrunk dynamic call graphs

1. A processor-implemented method for detecting an anomaly and analyzing a root cause of the anomaly in a real-time monitored application associated with a monitored system using a plurality of shrunk dynamic call graphs, the method comprising:

receiving, by the real-time monitored application associated with the monitored system, a plurality of input requests from a user input module;

automatically scanning, by an instrumentation agent, a plurality of lines of code of the real-time monitored application;

injecting an entry monitoring code and an exit monitoring code to each method associated with each class in the real-time monitored application;

generating, using the instrumentation agent, a plurality of shrunk dynamic call graphs for a plurality of input requests associated with the real-time monitored application by providing a shrink factor to the instrumentation agent, wherein the shrink factor is provided when the real-time monitored application receives the plurality of input requests, wherein the shrink factor is an amount of detail that is captured for each input request associated with the real-time monitored application;

storing the plurality of shrunk dynamic call graphs on a storage device;

transmitting the plurality of shrunk dynamic call graphs to an analysis engine over a network;

clustering, using an unsupervised machine learning algorithm by the analysis engine, the plurality of shrunk dynamic call graphs in to one or more groups of the plurality of shrunk dynamic call graphs based on an operation type;

detecting, using a machine learning model associated with the analysis engine, an anomalous group of the plurality of shrunk dynamic call graphs associated with a plurality of failed input requests from the one or more groups of the plurality of shrunk dynamic call graphs;

comparing, by the analysis engine, the anomalous group of the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests with the plurality of shrunk dynamic call graphs associated with the plurality of successful input requests to identify a deviation point in the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests; and

detecting, by the analysis engine, an anomaly of the real-time monitored application based on the deviation point in the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests and the exception thrown around the deviation point as the root cause of the identified anomaly in the real-time monitored application.

2. The method of claim 1 , wherein detecting the anomalous group of the plurality of shrunk dynamic call graphs comprises (i) obtaining a centroid of a dataset of each input request, (ii) obtaining a plurality of centroids of each of the one or more groups of the plurality of shrunk dynamic call graphs, (iii) obtaining the plurality of centroids based on the exception thrown in each input request, (iv) obtaining, using the plurality of centroids, a farthest centroid from the centroid of the dataset, (iv) identifying, using the farthest centroid, a first anomalous group of the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests, (v) identifying, using the farthest centroid, a second anomalous group based on the exceptions associated with the plurality of failed input requests, and (vi) identifying the anomalous group of the plurality of shrunk dynamic call graphs if the first anomalous group of the plurality of shrunk dynamic call graphs and the second anomalous group based on the exception are same.

3. The method of claim 1 , wherein the method comprises creating, using the machine learning model, the one or more groups of the plurality of shrunk dynamic call graphs based on a similarity between the plurality of shrunk dynamic call graphs after removing repeating patterns in the plurality of shrunk dynamic call graphs.

4. The method of claim 3 , wherein the one or more groups of the plurality of shrunk dynamic call graphs are created when (i) a distance between two points in each group is lower, and (ii) a minimum number of points are used to form a dense region in each group.

5. The method of claim 1 , wherein the plurality of shrunk dynamic call graphs comprises a crest point and a trough point with an absolute height.

6. The method of claim 5 , wherein the crest point or the trough point of the plurality of shrunk dynamic call graphs is generated when a difference between two adjacent points is greater than the shrink factor.

7. The method of claim 1 , wherein generating the plurality of shrunk dynamic call graphs comprises

(i) identifying a subroutine in the real-time monitored application that receives and handles the plurality of input requests;

(ii) inserting probes at each subroutine entry and each subroutine exit in the real-time monitored application;

(iii) tracking a running height, a running crest, and a running trough of the subroutine being entered;

(iv) obtaining the crest point in the plurality of shrunk dynamic call graphs by comparing an absolute difference of the running crest, and the running trough to the shrink factor;

(v) repeating the step (iii) and tracking the running height, the running crest, and the running trough of the subroutine being exited; and

(vi) obtaining the trough point in the plurality of shrunk dynamic call graphs by comparing an absolute difference of the running crest, and the running trough to the shrink factor.

8. A system for detecting an anomaly and analyzing a root cause of the anomaly in a real-time monitored application associated with a monitored system using a plurality of shrunk dynamic call graphs, comprising:

a processor; and

a non-transitory computer-readable storage medium storing one or more sequences of instructions, which when executed by the processor, causes to:

receive, by the real-time monitored application associated with the monitored system, a plurality of input requests from a user input module;

automatically scan, by an instrumentation agent, a plurality of lines of code of the real-time monitored application;

inject an entry monitoring code and an exit monitoring code to each method associated with each class in the real-time monitored application;

generate, using the instrumentation agent, a plurality of shrunk dynamic call graphs for a plurality of input requests associated with the real-time monitored application by providing a shrink factor to the instrumentation agent, wherein the shrink factor is provided when the real-time monitored application receives the plurality of input requests, wherein the shrink factor is an amount of detail that is captured for each input request associated with the real-time monitored application;

store the plurality of shrunk dynamic call graphs on a storage device;

transmit the plurality of shrunk dynamic call graphs to an analysis engine over a network;

cluster, using an unsupervised machine learning algorithm by the analysis engine, the plurality of shrunk dynamic call graphs in to one or more groups of the plurality of shrunk dynamic call graphs based on an operation type;

detect, using a machine learning model associated with the analysis engine, an anomalous group of the plurality of shrunk dynamic call graphs associated with a plurality of failed input requests from the one or more groups of the plurality of shrunk dynamic call graphs;

compare, by the analysis engine, the anomalous group of the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests with the plurality of shrunk dynamic call graphs associated with the plurality of successful input requests to identify a deviation point in the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests; and

detect, by the analysis engine, an anomaly of the real-time monitored application based on the deviation point in the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests and the exception thrown around the deviation point as the root cause of the identified anomaly in the real-time monitored application.

9. The system of claim 8 , wherein the processor is configured to detect the anomalous group of the plurality of shrunk dynamic call graphs by (i) obtaining a centroid of a dataset of each input request, (ii) obtaining a plurality of centroids of each of the one or more groups of the plurality of shrunk dynamic call graphs, (iii) obtaining the plurality of centroids based on the exception thrown in each input request, (iv) obtaining, using the plurality of centroids, a farthest centroid from the centroid of the dataset, (iv) identifying, using the farthest centroid, a first anomalous group of the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests, (v) identifying, using the farthest centroid, a second anomalous group based on the exceptions associated with the plurality of failed input requests, and (vi) identifying the anomalous group of the plurality of shrunk dynamic call graphs if the first anomalous group of the plurality of shrunk dynamic call graphs and the second anomalous group based on the exception are same.

10. The system of claim 8 , wherein the processor is configured to create using the machine learning model, the one or more groups of the plurality of shrunk dynamic call graphs based on a similarity between the plurality of shrunk dynamic call graphs after removing repeating patterns in the plurality of shrunk dynamic call graphs.

11. The system of claim 8 , wherein the processor is configured to create the one or more groups of the plurality of shrunk dynamic call graphs when (i) a distance between two points in the group is lower, and (ii) a minimum number of points to form a dense region in the group are essential while creating the group of the plurality of shrunk dynamic call graphs of a similar plurality of input requests.

12. The system of claim 8 , wherein the processor generates the plurality of shrunk dynamic call graphs by:

(i) identifying a subroutine in the real-time monitored application that receives and handles the plurality of input requests;

(ii) inserting probes at each subroutine entry and each subroutine exit in the real-time monitored application;

(iii) tracking a running height, a running crest, and a running trough of the subroutine being entered;

(iv) obtaining the crest point in the plurality of shrunk dynamic call graphs by comparing an absolute difference of the running crest, and the running trough to the shrink factor;

(v) repeating the step (iii) and tracking the running height, the running crest, and the running trough of the subroutine being exited; and

(vi) obtaining the trough point in the plurality of shrunk dynamic call graphs by comparing an absolute difference of the running crest, and the running trough to the shrink factor.

13. A non-transitory computer-readable storage medium storing the one or more sequence of instructions, which when executed by one or more processors, causes to perform a method for detecting an anomaly and analyzing a root cause of the anomaly in a real-time monitored application associated with a monitored system using a plurality of shrunk dynamic call graphs, by performing the steps of:

receiving, by the real-time monitored application associated with the monitored system, a plurality of input requests from a user input module;

automatically scanning, by an instrumentation agent, a plurality of lines of code of the real-time monitored application;

injecting an entry monitoring code and an exit monitoring code to each method associated with each class in the real-time monitored application;

generating, using the instrumentation agent, a plurality of shrunk dynamic call graphs for a plurality of input requests associated with the real-time monitored application by providing a shrink factor to the instrumentation agent, wherein the shrink factor is provided when the real-time monitored application receives the plurality of input requests, wherein the shrink factor is an amount of detail that is captured for each input request associated with the real-time monitored application;

storing the plurality of shrunk dynamic call graphs on a storage device;

transmitting the plurality of shrunk dynamic call graphs to an analysis engine over a network;

clustering, using an unsupervised machine learning algorithm by the analysis engine, the plurality of shrunk dynamic call graphs in to one or more groups of the plurality of shrunk dynamic call graphs based on an operation type;

detecting, using a machine learning model associated with the analysis engine, an anomalous group of the plurality of shrunk dynamic call graphs associated with a plurality of failed input requests from the one or more groups of the plurality of shrunk dynamic call graphs;

comparing, by the analysis engine, the anomalous group of the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests with the plurality of shrunk dynamic call graphs associated with the plurality of successful input requests to identify a deviation point in the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests; and

detecting, by the analysis engine, an anomaly of the real-time monitored application based on the deviation point in the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests and the exception thrown around the deviation point as the root cause of the identified anomaly in the real-time monitored application.

14. The non-transitory computer-readable storage medium storing the one or more sequence of instructions of claim 13 , wherein detecting the anomalous group of the plurality of shrunk dynamic call graphs comprises (i) obtaining a centroid of a dataset of each input request, (ii) obtaining a plurality of centroids of each of the one or more groups of the plurality of shrunk dynamic call graphs, (iii) obtaining the plurality of centroids based on the exception thrown in each input request, (iv) obtaining, using the plurality of centroids, a farthest centroid from the centroid of the dataset, (iv) identifying, using the farthest centroid, a first anomalous group of the plurality of shrunk dynamic call graphs associated with the plurality of failed input requests, (v) identifying, using the farthest centroid, a second anomalous group based on the exceptions associated with the plurality of failed input requests, and (vi) identifying the anomalous group of the plurality of shrunk dynamic call graphs if the first anomalous group of the plurality of shrunk dynamic call graphs and the second anomalous group based on the exception are same.

15. The non-transitory computer-readable storage medium storing the one or more sequence of instructions of claim 13 , wherein the method further includes creating, using the machine learning model, the one or more groups of the plurality of shrunk dynamic call graphs based on a similarity between the plurality of shrunk dynamic call graphs after removing repeating patterns in the plurality of shrunk dynamic call graphs.

16. The non-transitory computer-readable storage medium storing the one or more sequence of instructions of claim 13 , wherein generating the plurality of shrunk dynamic call graphs comprises,

(i) identifying a subroutine in the real-time monitored application that receives and handles the plurality of input requests;

(ii) inserting probes at each subroutine entry and each subroutine exit in the real-time monitored application;

(iii) tracking a running height, a running crest, and a running trough of the subroutine being entered;

(iv) obtaining the crest point in the plurality of shrunk dynamic call graphs by comparing an absolute difference of the running crest, and the running trough to the shrink factor;

(v) repeating the step (iii) and tracking the running height, the running crest, and the running trough of the subroutine being exited; and

(vi) obtaining the trough point in the plurality of shrunk dynamic call graphs by comparing an absolute difference of the running crest, and the running trough to the shrink factor.