Identifying and remediating anomalies in a self-healing network
Some embodiments of the invention provide a method for remediating anomalies in an SD-WAN implemented by multiple forwarding elements (FEs) located at multiple sites connected by the SD-WAN. The method is performed for each particular FE in a set of one or more FEs. The method identifies a set of metrics associated with each application of multiple applications for which the particular FE forwards traffic flows. For each particular application of the multiple applications, the method generates a distribution graph that shows the identified set of metrics associated with the particular application for the particular FE over a first duration of time. The method analyzes the generated distribution graphs using a machine-trained process to identify one or more per-application incidents by identifying that a threshold number of metrics associated with the particular application (1) are outliers with respect to the generated distribution graph for the particular application and (2) occurred within a second duration of time.
1 . A method of remediating anomalies in an SD-WAN (software-defined wide-area network) implemented by a plurality of forwarding elements (FEs) located at a plurality of sites connected by the SD-WAN, the method comprising:
for each particular FE in a set of one or more FEs:
identifying a set of metrics associated with each application in a plurality of applications for which the particular FE forwards traffic flows;
for each particular application in the plurality of applications, generating a distribution graph that shows the identified set of metrics associated with the particular application for the particular FE over a first duration of time;
analyzing the generated distribution graphs, for the particular application across multiple FEs to determine whether an identified per-application incident on one particular FE is detected in the multiple FEs, using a machine-trained process to identify one or more per-application incidents by identifying that a threshold number of metrics associated with the particular application (i) are outliers with respect to the generated distribution graph for the particular application and (ii) occurred within a second duration of time; and
applying remedial action for the outliers by adjusting communication hardware configuration.
2 . The method of claim 1 , wherein said identifying a set of metrics, generating, and analyzing are performed by at least one anomaly detecting process executing on at least one host computer.
3 . The method of claim 2 , wherein the SD-WAN is a self-healing SD-WAN that includes one or more such anomaly detecting processes that (i) identify remedial actions for remediating anomalous behavior associated with the identified outliers and (ii) implement the identified remedial actions in the SD-WAN.
4 . The method of claim 1 , wherein each set of metrics comprises metrics for the particular FE per-application, per-path through the SD-WAN.
5 . The method of claim 1 , wherein each identified set of metrics comprises a set of collected operational values.
6 . The method of claim 5 , wherein each set of identified metrics comprises a set of performance scores computed based on the set of collected operational values.
7 . The method of claim 6 , wherein the particular FE comprises one of a plurality of edge FEs, wherein performance scores are computed for each edge FE in the plurality of edge FEs per-application, per-path.
8 . The method of claim 6 , wherein generating a distribution graph for each particular application in the plurality of applications comprises generating a distribution graph that represents a distribution of the set of performance scores for the particular application over the first duration of time.
9 . The method of claim 8 , wherein the first duration of time comprises 30 minutes, wherein each performance score represents application performance by the particular FE for each minute of the 30 minute duration of time.
10 . The method of claim 8 , wherein generating a distribution graph for each particular application in the plurality of applications further comprises, for the set of performance scores for the particular application, computing (i) a sample mean for the first duration of time and (ii) a standard deviation for the first duration of time.
11 . The method of claim 10 , wherein the sample mean and the standard deviation comprise dynamic parameters that change over time based on the generated performance scores.
12 . The method of claim 11 , wherein identifying that a threshold number of metrics associated with the particular application are outliers with respect to the generated distribution graph for the particular application comprises using the dynamic parameters to determine whether a threshold number of performance scores in the set of performance scores for the particular application exceed a specified threshold of performance.
13 . The method of claim 1 , wherein each particular application (i) executes on a device operating at one of the plurality of sites and (ii) uses the SD-WAN's FEs to forward traffic flows to another location connected to the SD-WAN.
14 . The method of claim 13 , wherein: the other location is another site or datacenter connected to the SD-WAN, and the datacenter comprises a private datacenter or public datacenter.
15 . The method of claim 1 , wherein each generated distribution graph comprises a distribution curve.
16 . The method of claim 15 , wherein the machine-trained process comprises a sliding-window Gaussian outlier detection machine-trained process and the distribution curve comprises a Gaussian distribution curve.
17 . The method of claim 1 , wherein the second duration of time is different than the first duration of time.
18 . The method of claim 1 , wherein the second duration of time is a same duration of time as the first duration of time.
19 . The method of claim 1 , wherein the second duration of time is a subset of the first duration of time.
20 . The method of claim 1 , wherein the plurality of FEs comprise a plurality of routers.