Autonomous distributed forwarding plane traceability based anomaly detection in application traffic for hyper-scale SD-WAN
Some embodiments of the invention provide a method for detecting and remediating anomalies in an SD-WAN that includes a controller, at least one enterprise datacenter, and multiple branch sites each having at least one edge node that includes a set of packet processing stages. At a particular node in the SD-WAN, the method receives, from the controller, trace monitoring rules specified for a particular packet flow. The method determines that a first packet received at the particular node belongs to the particular packet flow and matches at least one of the trace monitoring rules. Based on the determination, the method specifies the first packet as a packet that should be trace monitored by each packet processing stage of the particular node. As the first packet is processed by the set of packet processing stages, the method generates trace monitoring results to be provided to the controller for analysis.
1. A method for detecting and remediating anomalies in a software-defined wide area network (SD-WAN) comprising a controller, at least one enterprise datacenter, and a plurality of branch sites, wherein each branch site comprises at least one forwarding node, each forwarding node comprising a set of packet processing stages that process packet flows that are forwarded through the SD-WAN, the method comprising:
at a particular forwarding node that comprises a plurality of packet processing stages:
receiving, from the controller of the SD-WAN, a set of one or more trace monitoring rules specified for a particular packet flow;
determining that packets of the particular packet flow received at the particular forwarding node match at least one trace monitoring rule in the set of trace monitoring rules;
based on the determination, directing each packet processing stage in the plurality of packet processing stages of the particular forwarding node to perform trace monitoring for the packets of the particular packet flow; and
forwarding a plurality of trace monitoring results for the particular packet flow to the controller for analysis,
wherein the plurality of packet processing stages comprise one or more of: ingress, firewall, routing, quality of service (QoS), network address translation (NAT), and egress.
2. The method of claim 1 , wherein the ingress packet processing stage (i) determines that a set of one or more packets of the particular packet flow received at the particular forwarding node belongs to the particular packet flow and matches at least one trace rule in the set of trace rules and (ii) specifies the set of packets as packets that should be trace monitored.
3. The method of claim 1 , wherein specifying the set of packets as packets that should be trace monitored further comprises setting a flag on each of the set of packets.
4. The method of claim 1 further comprising receiving a second set of one or more trace monitoring rules specified for one of (i) packet flows for a particular application, (ii) packet flows between a specified set of forwarding nodes in the plurality of forwarding nodes, and (iii) all packet flows between the plurality of forwarding nodes in the SD-WAN.
5. The method of claim 1 , wherein the set of one or more trace monitoring rules further comprise a set of exception rules, wherein each exception rule defines a threshold value for identifying anomalous behavior on the particular forwarding node.
6. The method of claim 1 further comprising:
receiving a particular packet and determining that the particular packet does not belong to the particular packet flow; and
processing the particular packet without performing trace monitoring on the particular packet.
7. A method for detecting and remediating anomalies in a software-defined wide area network (SD-WAN) comprising a controller, at least one enterprise datacenter, and a plurality of branch sites, wherein each branch site comprises at least one forwarding node, each forwarding node comprising a set of packet processing stages that process packet flows that are forwarded through the SD-WAN, the method comprising:
at a trace monitor agent of a particular forwarding node that comprises a plurality of packet processing stages:
receiving, from the controller of the SD-WAN, a set of one or more trace monitoring rules specified for a particular packet flow;
determining that packets of a particular packet flow received at the particular forwarding node match at least one trace monitoring rule in the set of trace monitoring rules;
based on the determination, directing each packet processing stage in the plurality of packet processing stages of the particular forwarding node to perform trace monitoring for the packets of the particular packet flow, wherein each packet processing stage in the plurality of packet processing stages generates trace monitoring results for the particular packet flow and provides the generated trace monitoring results to the trace monitor agent of the particular forwarding node; and
forwarding a plurality of trace monitoring results for the particular flow to the controller for analysis,
wherein the plurality of packet processing stages comprise one or more of: ingress, firewall, routing, quality of service (QoS), network address translation (NAT), and egress.
8. The method of claim 7 , wherein the set of one or more trace monitoring rules specifies a number of packets to be traced, wherein the trace monitor agent provides the generated trace monitoring results to the controller when the specified number of packets have been traced.
9. The method of claim 8 , wherein the trace monitor agent provides the generated trace monitoring results to the controller via a control plane executing on the particular forwarding node.
10. The method of claim 8 , wherein the controller (i) analyzes the provided trace monitoring results and (ii) performs a dynamic action when an anomaly is identified, wherein the dynamic action auto-corrects the identified anomaly.
11. A non-transitory machine readable medium storing a program for execution by a set of processing units, the program for detecting and remediating anomalies in a software-defined wide area network (SD-WAN) comprising a controller, at least one enterprise datacenter, and a plurality of branch sites, wherein each branch site comprises at least one forwarding node, each forwarding node comprising a set of packet processing stages that process packet flows that are forwarded through the SD-WAN, the program comprising sets of instructions for:
at a particular forwarding node that comprises a plurality of packet processing stages:
receiving, from the controller of the SD-WAN, a set of one or more trace monitoring rules specified for a particular packet flow;
determining that packets of the particular packet flow received at the particular forwarding node match at least one trace monitoring rule in the set of trace monitoring rules;
based on the determination, directing each packet processing stage in the plurality of packet processing stages of the particular forwarding node to perform trace monitoring for the packets of the particular packet flow; and
forwarding a plurality of trace monitoring results for the particular packet flow to the controller for analysis,
wherein the plurality of packet processing stages comprise one or more of: ingress, firewall, routing, quality of service (QoS), network address translation (NAT), and egress.
12. The non-transitory machine readable medium of claim 11 , wherein the ingress packet processing stage (i) determines that a set of one or more packets of the particular packet flow received at the particular forwarding node belongs to the particular packet flow and matches at least one trace rule in the set of trace rules and (ii) specifies the set of packets as packets that should be trace monitored.
13. The non-transitory machine readable medium of claim 11 , wherein specifying the set of packets as packets that should be trace monitored further comprises setting a flag on each of the set of packets.
14. The non-transitory machine readable medium of claim 11 , wherein each packet processing stage in the plurality of packet processing stages generates trace monitoring results for the particular packet flow and provides the generated trace monitoring results to a trace monitor agent executing on the particular forwarding node.
15. The non-transitory machine readable medium of claim 14 , wherein the set of one or more trace monitoring rules specifies a number of packets to be traced, wherein the trace monitor agent provides the generated trace monitoring results to the controller when the specified number of packets have been traced.
16. The non-transitory machine readable medium of claim 15 , wherein the trace monitor agent provides the generated trace monitoring results to the controller via a control plane executing on the particular forwarding node.
17. The non-transitory machine readable medium of claim 15 , wherein the controller (i) analyzes the provided trace monitoring results and (ii) performs a dynamic action when an anomaly is identified, wherein the dynamic action auto-corrects the identified anomaly.
18. The non-transitory machine readable medium of claim 11 further comprising receiving a second set of one or more trace monitoring rules specified for one of (i) packet flows for a particular application, (ii) packet flows between a specified set of forwarding nodes in the plurality of forwarding nodes, and (iii) all packet flows between the plurality of forwarding nodes in the SD-WAN.
19. The non-transitory machine readable medium of claim 11 , wherein the set of one or more trace monitoring rules further comprise a set of exception rules, wherein each exception rule defines a threshold value for identifying anomalous behavior on the particular forwarding node.
20. The non-transitory machine readable medium of claim 11 further comprising:
receiving a particular packet and determining that the particular packet does not belong to the particular packet flow; and
processing the particular packet without performing trace monitoring on the particular packet.