Filter with engineered damping for load-balanced fine-grained adaptive routing in high-performance system interconnect
A switch is provided for routing packets in an interconnection network. The switch includes a plurality of egress ports to transmit packets. The switch also includes one or more ingress ports to receive packets. The switch also includes a port and bandwidth capacity circuit configured to obtain (i) port capacity for a plurality of egress ports of the switch, and (ii) bandwidth capacity for transmitting packets to a destination. The switch also includes a network capacity circuit configured to compute network capacity, for transmitting packets to the destination, via the plurality of egress ports, based on a function of the port capacity and the bandwidth capacity. The switch also includes a routing circuit configured to route one or more packets received via one or more ingress ports of the switch, to the destination, via the plurality of egress ports, based on the network capacity.
1. A switch for routing packets in an interconnection network, the switch comprising:
a plurality of egress ports to transmit packets;
one or more ingress ports to receive packets;
a port and bandwidth capacity circuit configured to obtain (i) port capacity for a plurality of egress ports of the switch, and (ii) bandwidth capacity for transmitting packets to a destination, wherein the bandwidth capacity comprises configured buffer minus current buffer in a next switch;
a network capacity circuit configured to compute network capacity, for transmitting packets to the destination, via the plurality of egress ports, based on a function of the port capacity and the bandwidth capacity; and
a routing circuit configured to route one or more packets received via one or more ingress ports of the switch, to the destination, via the plurality of egress ports, based on the network capacity.
2. The switch of claim 1 , wherein the port capacity comprises available buffer capacity for ingress ports of respective receiver switches coupled to the plurality of egress ports.
3. The switch of claim 1 , wherein the port capacity is zero through any egress port that has a fault.
4. The switch of claim 1 , wherein the bandwidth capacity comprises idle buffer in a next switch.
5. The switch of claim 1 , wherein the bandwidth capacity is calculated based on one or more telemetry packets received from another switch of the interconnection network.
6. The switch of claim 1 , wherein the function comprises Exponential Weighted Moving Average.
7. The switch of claim 1 , wherein the function comprises a plurality of low-pass filters.
8. The switch of claim 7 , wherein each low-pass filter is configured to combine the port capacity for a respective egress port with the bandwidth capacity, to obtain a respective bandwidth capacity for transmitting packets to the destination via the respective egress port.
9. The switch claim 1 , wherein:
the switch is connected to the next switch using a plurality of virtual lanes; the bandwidth capacity includes a respective buffer capacity for each virtual lane;
the network capacity circuit is configured to compute, for each virtual lane, a respective virtual lane capacity, using a respective one or more low-pass filters, based on the port capacity and the respective buffer capacity; and
the routing circuit is configured to route the one or more packets to the destination by selecting a virtual lane from the plurality of virtual lanes based on the respective virtual lane capacity.
10. The switch of claim 1 , wherein the bandwidth capacity includes idle buffers in a path to the destination that includes an intermediate switch.
11. The switch of claim 1 , wherein:
the bandwidth capacity includes (i) a first buffer capacity corresponding to idle buffers in a first path to the destination via a first intermediate switch, and (ii) a second buffer capacity corresponding to idle buffers in a second path to the destination via a second intermediate switch;
the network capacity circuit is configured to:
compute a first network capacity for transmitting packets to the destination, via a first port, using a low-pass filter, based on the port capacity and the first buffer capacity; and
compute a second network capacity for transmitting packets to the destination, via the plurality of egress ports, using a second low-pass filter, based on the port capacity and the first buffer capacity; and
the routing circuit is configured to route the one or more packets by selecting between the first path and the second path, based on the first network capacity and the second network capacity.
12. A switch for routing packets in an interconnection network, the switch comprising:
a plurality of egress ports to transmit packets;
one or more ingress ports to receive packets;
a port and bandwidth capacity circuit configured to obtain (i) port capacity for a plurality of egress ports of the switch, and (ii) bandwidth capacity for transmitting packets to a destination;
a network capacity circuit configured to compute network capacity, for transmitting packets to the destination, via the plurality of egress ports, based on a function of the port capacity and the bandwidth capacity, wherein the function comprises a plurality of low-pass filters, and wherein each low-pass filter is configured to combine the port capacity for a respective egress port with the bandwidth capacity, to obtain a respective bandwidth capacity for transmitting packets to the destination via the respective egress port; and
a routing circuit configured to route one or more packets received via one or more ingress ports of the switch, to the destination, via the plurality of egress ports, based on the network capacity.
13. The switch of claim 12 , wherein the port capacity comprises available buffer capacity for ingress ports of respective receiver switches coupled to the plurality of egress ports.
14. The switch of claim 12 , wherein the port capacity is zero through any egress port that has a fault.
15. The switch of claim 12 , wherein the bandwidth capacity comprises idle buffer in a next switch.
16. The switch of claim 12 , wherein the bandwidth capacity comprises configured buffer minus current buffer in a next switch.
17. A switch for routing packets in an interconnection network, the switch comprising:
a plurality of egress ports to transmit packets;
one or more ingress ports to receive packets;
a port and bandwidth capacity circuit configured to obtain (i) port capacity for a plurality of egress ports of the switch, and (ii) bandwidth capacity for transmitting packets to a destination;
a network capacity circuit configured to compute network capacity, for transmitting packets to the destination, via the plurality of egress ports, based on a function of the port capacity and the bandwidth capacity; and
a routing circuit configured to route one or more packets received via one or more ingress ports of the switch, to the destination, via the plurality of egress ports, based on the network capacity,
wherein:
the switch is connected to the next switch using a plurality of virtual lanes; the bandwidth capacity includes a respective buffer capacity for each virtual lane;
the network capacity circuit is configured to compute, for each virtual lane, a respective virtual lane capacity, using a respective one or more low-pass filters, based on the port capacity and the respective buffer capacity; and
the routing circuit is configured to route the one or more packets to the destination by selecting a virtual lane from the plurality of virtual lanes based on the respective virtual lane capacity.
18. The switch of claim 17 , wherein the bandwidth capacity is calculated based on one or more telemetry packets received from another switch of the interconnection network.
19. The switch of claim 17 , wherein the function comprises Exponential Weighted Moving Average.
20. The switch of claim 17 , wherein the function comprises a plurality of low-pass filters.