Reducing internal fabric congestion in leaf-spine switch fabric
View Patent ↗Embodiments of the present invention provide methods and system to reduce needless data traffic in leaf-spine switch fabric. In embodiments, in a static solution, data rates of data flows having a common destination port may be measured and responsive to the data flows having a combined data rate that exceeding a rate limit of the common destination port, one or more of the data flows may be limited. Embodiments may also comprise a global scheduler to provide dynamic data rate controls of traffic flows from source ports to destination ports in which to reduce the handling of data traffic that would otherwise be discarded due to oversubscription.
1. An information handling system (IHS) comprising:
a plurality of ports for sending data to and receiving data t-o from one or more devices; one or more processors that are communicatively coupled to the plurality of ports to facilitate the sending and receiving of data; and
a data rate measurer, executed by one or more processors, that measures data rates of one or more data flows received at one or more ports from the plurality of ports, the one or more data flows having a common destination port in a leaf-spine system; and
a rate limiter, executed by one or more processors, that responsive to a combined data rate of the one or more data flows exceeding a rate limit of the common destination port at a second IHS in the leaf-spine system, limits egress of the one or more data flows from the IHS based on the rate limit of the common destination port.
2. The information handling system (IHS) of claim 1 , wherein the IHS is a leaf node in the leaf-spine system and wherein the common destination port is at a second leaf node in the leaf-spine system.
3. The information handling system (IHS) of claim 2 , wherein a rate limit of each port of a set of ports at the second leaf node are communicated to the IHS.
4. The information handling system (IHS) of claim 1 , wherein the IHS is a spine node in the leaf-spine system, the one or more ports are at one or more leaf nodes in the leaf-spine system, and the second IHS is a second leaf node in the leaf-spine system.
5. The information handling system (IHS) of claim 4 , wherein at least two data flows from the one or more data flows ingressing to the IHS from different leaf nodes in the leaf-spine system routed to the common destination port via the spine node.
6. The information handling system (IHS) of claim 1 , wherein a rate limit of each port of a set of ports at the second IHS is communicated to the IHS.
7. The information handling system (IHS) of claim 1 , wherein the rate limit of the common destination port comprises a port speed of the common destination port.
8. The information handling system (IHS) of claim 7 , wherein the rate limit of the common destination port further comprises the port speed plus a burst size.
9. The information handling system (IHS) of claim 8 , wherein the burst size is related to at least a buffer size of the second IHS.
10. A method of dynamic rate limiting, the method comprising: receiving data rates of traffic flows from source ports to a common destination port having a rate limit;
calculating an allocated rate for each of the traffic flows based on the number of the traffic flows, the data rates, and the rate limit, wherein each of the data rates corresponds to one of the traffic flows from a source port among the source ports to the common destination port; and
for each traffic flow that has a data rate that exceeds its allocated rate, sending the allocated rate to a dynamic rate limiter residing on a leaf node in a leaf-spine system hosting the source port for that traffic flow.
11. The method of claim 10 , wherein the calculating uses a max-min fairness algorithm.
12. The method of claim 11 , wherein the max-min fairness algorithm includes the steps of:
(a) dividing the rate limit by the number of the traffic flows to obtain an average;
(b) for a data rate in the data rates, the data rate corresponding to a traffic flow,
responsive to the data rate not exceeding the average, setting an allocated rate corresponding to the traffic flow to the data rate, removing the data rate from the data rates, removing the allocated rate from the set of allocated rates, subtracting the allocated rate from the rate limit, and reducing the number of the traffic flows by one;
responsive to the data rate exceeding the average, adding the average to the allocated rate corresponding to the traffic flow, subtracting the average from the data rate, and subtracting the average from the rate limit; and
(c) repeating steps (a) to (b) while the number of the traffic flows and the rate limit are positive.
13. The method of claim 10 , wherein the calculating includes:
identifying strict priority quality-of-service (QoS) classes and non-strict priority QoS classes in the traffic flows;
allocating a bandwidth to each of the strict priority QoS classes based on the rate limit and a corresponding data rate in the data rates;
calculating a leftover bandwidth by subtracting the bandwidth from the rate limit; and
allocating the leftover bandwidth to the non-strict priority QoS classes by weight.
14. The method of claim 13 , wherein allocating the leftover bandwidth includes:
allocating the leftover bandwidth to the non-strict priority QoS classes by weight;
calculating an excess bandwidth based on the allocated bandwidth to undersubscribed classes among the non-strict priority QoS classes and the leftover bandwidth; and
allocating the excess bandwidth to oversubscribed classes among the non-strict priority QoS classes.
15. A global scheduler system for rate limiting data flows, the global scheduler system comprising:
a rate monitor, executed by one or more processors, that receives data rates of traffic flows from source ports to a common destination port having a rate limit;
a rate allocator, executed by one or more processors, to calculate an allocated rate for each of the traffic flows based on the number of the traffic flows, the data rates, and the rate limit, wherein each of the data rates corresponds to one of the traffic flows from a source port among the source ports to the common destination port; and
a rate limit notifier, executed by one or more processors, that for each traffic flow that has a data rate that exceeds its allocated rate, sends the allocated rate to a dynamic rate limiter residing on a leaf node in a leaf-spine system hosting the source port for that traffic flow.
16. The system of claim 15 , further comprising:
a leaf rate limiter, residing on each of leaf nodes hosting the source ports, being responsive to a data rate exceeding the rate limit of the common destination port, configured to limit the traffic flows to the common destination port based on the rate limit of the common destination port.
17. The system of claim 15 , further comprising:
a spine node, configured to measure a data rate of at least one traffic flow among the traffic flows switched through the spine node to the common destination port; and
a spine rate limiter residing on the spine node, being responsive to the data rate exceeding the rate limit of the common destination port, configured to limit the at least one traffic flow to the common destination port based on the rate limit of the common destination port.
18. The system of claim 15 , wherein the global scheduler is configured to evaluate the common destination port to identify an oversubscribed destination port, and allocate among a set of traffic flows to the oversubscribed destination port.
19. The system of claim 15 , wherein the at least one leaf node includes Access Control List (ACL) counters indicative of the traffic flows between active source ports on the at least one leaf node and the common destination node, each of the active source ports includes at least one of an active source port on a leaf node.
20. The system of claim 15 , wherein the dynamic rate limiter includes:
a priority-based flow control (PFC) issuer to issue a PFC message on the source port when a source data rate of the source port exceeds the allocated rate.