Monitoring a multi-tier network fabric
The following description is directed to monitoring a multi-tier network including a first tier of routers and a second tier of routers. In one embodiment, a method of monitoring the network comprises creating a probe packet. The probe packet can include a source address corresponding to a monitoring function, and a destination address served by a path from the first tier to the second tier of routers. The probe packet can be transmitted into the multi-tier network. A number of probe packets counted by each router of the first and second tiers of routers can be received. One or more performance metrics of the multi-tier network can be determined based, at least in part, on a sum of the number of probe packets counted by the first tier of routers and a sum of the number of probe packets counted by the second tier of routers.
1. A system for routing network traffic, the system comprising:
a multi-tier network including a first tier of routers connected to a second tier of routers, the multi-tier network including multiple paths between the first tier of routers and the second tier of routers, the first tier of routers and the second tier of routers being neighboring tiers, each of the routers of the first and second tiers including:
load balancing logic configured to spread packets across the multiple paths to routers of the neighboring tier;
an access control list (ACL) rule for matching probe packets having a source address field within a predetermined range; and
a counter triggered on the ACL rule being matched;
a first processor executing probing agent software configured to transmit a probe packet including a destination address served by multiple paths from the first tier of routers to the second tier of routers and a source address field having a false source address identifying the packet as a probe packet, the false source address being within the predetermined range; and
a second processor executing collector agent software connected to the multi-tier network and configured to:
receive a value of the counter of each router of the first and second tiers of the multi-tier network, the value of the counter corresponding to a number of probe packets counted at each router, and
determine one or more performance metrics of a particular path through the multi-tier network based, at least in part, on a sum of the number of probe packets counted at respective routers of the first tier of routers and a sum of the number of probe packets counted at respective routers of the second tier of routers.
2. The system of claim 1 , wherein the second processor is further configured to determine that the first tier of routers is degraded if the number of probe packets received at the second tier of routers is less than the number of probe packets received at the first tier of routers.
3. The system of claim 1 , wherein the multi-tier network is a multi-tier Clos network fabric using equal cost multipath (ECMP) routing between the first tier of routers and the second tier of routers, and determining the one or more performance metrics of the particular path through the multi-tier network includes determining that a first router of the first tier of routers is degraded when the number of probe packets counted at the first router differs by more than a predetermined threshold from the number of probe packets counted at each of the other routers of the first tier of routers.
4. The system of claim 1 , wherein each router of the first tier of routers and each router of the second tier of routers includes a ternary content addressable memory (TCAM) for storing an ACL table.
5. The system of claim 1 , wherein the transmitted probe packet includes a time-to-live field with a value less than or equal to a number of tiers of the multi-tier network.
6. The system of claim 1 , wherein the destination address of the probe packet corresponds to an unused address within a subnet of an edge router of the multi-tier network.
7. The system of claim 1 , wherein the multi-tier network comprises an edge tier, and each router of the edge tier includes an ACL rule for dropping probe packets having a source address field within the predetermined range.
8. A method of monitoring a multi-tier network including a first tier of routers and a second tier of routers, the method comprising:
creating, by a host computer, a probe packet comprising:
a source address field having a value corresponding to a forged source address identifying the packet as a probe packet, and
a destination address served by a path from the first tier of routers to the second tier of routers;
transmitting the probe packet from the host computer and into the multi-tier network, wherein the multi-tier network comprises multiple paths between the first tier of routers and the second tier of routers, respective routers of the first tier of routers and the second tier of routers configured to spread packets across the multiple paths between the first tier of routers and the second tier of routers;
receiving a number of probe packets counted by each router of the first tier of routers and each router of the second tier of routers; and
determining one or more performance metrics of a particular tier of the multi-tier network based, at least in part, on a difference between a sum of the number of probe packets counted by the routers of the first tier of routers and a sum of the number of probe packets counted by the routers of the second tier of routers.
9. The method of claim 8 , wherein the source address field is within a predetermined range that matches an access control list (ACL) rule of each router of the first tier of routers and each router of the second tier of routers.
10. The method of claim 8 , wherein the source address field of the probe packet is within a first predetermined range when the probe packet is transmitted during a first time period, and the source address field of the probe packet is within a second predetermined range when the probe packet is transmitted during a second time period.
11. The method of claim 8 , wherein determining the one or more performance metrics of the particular tier of the multi-tier network includes determining that the first tier of routers is degraded when the number of probe packets received at the second tier of routers is less than the number of probe packets received at the first tier of routers.
12. The method of claim 8 , wherein the multi-tier network uses equal cost multipath (ECMP) routing between the first tier of routers and the second tier of routers, and determining the one or more performance metrics of the particular tier of the multi-tier network includes determining that a first router of the first tier of routers is degraded when the number of probe packets counted at the first router differs from the number of probe packets counted at each of the other routers of the first tier of routers by more than a predetermined threshold.
13. The method of claim 8 , wherein the transmitted probe packet includes a time-to-live field with a value that prevents the probe packet from reaching the destination address.
14. The method of claim 8 , wherein the destination address of the probe packet corresponds to an unused address within a subnet of an edge router of the multi-tier network.
15. A computer-readable storage medium including instructions thereon for executing a method of monitoring a multi-tier network including a first tier of routers and a second tier of routers, the method comprising:
creating a group of probe packets, each probe packet comprising:
a source address field having a value representing a false source address and identifying the packet as a probe packet, the value different than a network address of a source node of the probe packet, and
a destination address corresponding to an address reachable by a router of an edge tier of the multi-tier network;
transmitting the group of probe packets into the multi-tier network, wherein the multi-tier network comprises multiple paths between the first tier of routers and the second tier of routers, respective routers of the first tier of routers and the second tier of routers configured to spread packets across the multiple paths between the first tier of routers and the second tier of routers;
waiting for the group of probe packets to traverse the multi-tier network and get dropped at the edge of the multi-tier network and then querying each router of the first tier of routers and each router of the second tier of routers for a number of probe packets counted by each router; and
determining one or more performance metrics of a particular tier of the multi-tier network based, at least in part, on a difference between a sum of the number of probe packets counted by the routers of the first tier of routers and a sum of the number of probe packets counted by the routers of the second tier of routers.
16. The computer-readable storage medium of claim 15 , wherein each probe packet of the group of probe packets includes a different source port.
17. The computer-readable storage medium of claim 15 , wherein the source address field of each probe packet corresponds to a port of a router of the first or second tier of routers, and querying for the number of probe packets includes querying for the number of probe packets associated with each port of the router.
18. The computer-readable storage medium of claim 15 , wherein the multi-tier network uses equal cost multipath (ECMP) routing between the first tier of routers and the second tier of routers, and determining the one or more performance metrics of the particular tier of the multi-tier network includes determining that a first router of the first tier of routers is degraded when the number of probe packets counted at the first router differs from the number of probe packets counted at each of the other routers of the first tier of routers by more than a predetermined threshold.
19. The computer-readable storage medium of claim 15 , wherein the source address field of the probe packet is within a first predetermined range when the probe packet is transmitted during a first time period, and the source address field of the probe packet is within a second predetermined range when the probe packet is transmitted during a second time period.
20. The computer-readable storage medium of claim 15 , wherein each transmitted probe packet of the group of probe packets includes a time-to-live field with a value that prevents the probe packet from reaching the destination address.