Scalable, low latency, deep buffered switch architecture
View Patent ↗A switch architecture includes an ingress module, ingress fabric interface module, and a switch fabric. The switch fabric communicates with egress fabric interface modules and egress modules. The architecture implements multiple layers of congestion management. The congestion management may include fast acting link level flow control and more slowly acting end-to-end flow control. The switch architecture simultaneously provides high scalability, with low latency and low frame loss.
1. A device, comprising:
packet flow architecture components comprising:
a plurality of ingress fabric interfaces;
a switching fabric in communication with the plurality of ingress fabric interfaces;
a plurality of egress fabric interfaces in communication with the switching fabric;
an egress module in communication with the plurality of egress fabric interfaces; and
flow control circuitry configured to:
communicate a link level (LL) flow control message from an originating architecture component to a link partner among the packet flow architecture components, the LL flow control message being separate from data packets communicated by the device and the LL flow control message including a protocol frame that includes bits indicating congestion for each class of service (CoS); and
after communicating the LL flow control message, remove blocking resulting from the LL flow control message by transmitting an end-to-end (EE) flow control message from the egress module to the switching fabric and then the switching fabric multicasting the EE flow control message to the plurality of ingress fabric interfaces, wherein
each of the ingress fabric interfaces is a separate component from each of the egress fabric interfaces,
input data flows from an external source, through an ingress port, to an ingress fabric interface and then to the switching fabric,
output data flows from the switching fabric to an egress fabric interface and then, through an egress port, to an external destination,
the EE flow control message is transmitted directly from the egress module to the switching fabric without passing through any of the plurality of egress fabric interfaces,
the EE flow control message specifies a particular CoS and physical egress port, and
the blocking comprises head of line blocking.
2. The device of claim 1 , wherein
each ingress fabric interface comprises a first packet queue organized by physical egress port and CoS, and
the originating architecture component is configured to respond to an input port and CoS counter by communicating the LL flow control message.
3. The device of claim 1 , wherein the LL flow control message comprises a specifier of a congested input port within the originating architecture component.
4. The device of claim 1 , wherein the LL flow control message comprises a specifier of both a congested input port and CoS within the originating architecture component.
5. The device of claim 1 , wherein the EE flow control message comprises a specifier of a congested queue for the physical egress port handled by the egress module.
6. The device of claim 1 , wherein the EE flow control message comprises a specifier of a congested queue by the particular CoS and the physical egress port handled by the egress module.
7. The device of claim 1 , wherein the LL flow control message, the EE flow control message, or both comprise a queue stop message.
8. The device of claim 1 , wherein the LL flow control message, the EE flow control message, or both comprise a queue restart message.
9. The device of claim 1 , wherein the LL flow control message and the EE flow control message specify congestion at different levels of granularity.
10. The device of claim 1 , wherein the flow control is operable to perform EE flow control and LL flow control together to avoid packet loss.
11. The device of claim 1 , wherein the link partner comprises a physically adjacent architecture component with respect to the originating architecture component.
12. The device of claim 1 , wherein the LL flow control message is configured to pass through the egress fabric interface to the link partner.
13. The device of claim 1 , wherein
first data packets received by the device are received via at least one ingress fabric interface, and
second data packets transmitted by the device are transmitted via at least one egress fabric interface.
14. A device, comprising:
a modularized packet routing architecture comprising:
ingress modules assigned to ingress fabric interfaces;
egress modules assigned to egress fabric interfaces;
a fabric interface between the ingress fabric interfaces and the egress fabric interfaces;
link level (LL) flow control circuitry executing contemporaneously with end-to-end (EE) flow control circuitry; and
the EE flow control circuitry, wherein
each of the ingress fabric interfaces is a separate component from each of the egress fabric interfaces,
input data flows from an external source, through an ingress port, to an ingress module, then to a corresponding ingress fabric interface and then to the switching fabric,
output data flows from the switching fabric to an egress fabric interface, then to a corresponding egress module and then, through an egress port, to an external destination,
the LL flow control circuitry is configured to execute LL flow control on input ports of link partners in the modularized packet routing architecture via LL flow control messaging that is separate from data packets communicated by the device, the LL flow control messaging including a protocol frame that includes bits indicating congestion for each class of service (CoS),
the EE flow control circuitry is configured to alleviate blocking resulting from the LL flow control by transmitting EE flow control messages, from the egress modules to the fabric interface and then the fabric interface multicasting the EE flow control messages to the ingress modules,
the EE flow control messages are transmitted directly from the egress modules to the fabric interface without passing through any of the plurality of egress fabric interfaces, and
the EE flow control messages each specify a CoS and corresponding physical egress port.
15. The device of claim 14 , wherein
the EE flow control circuitry is configured to transmit the EE flow control messages across multiple link partners by communicating the EE flow control messages from an egress module among the egress modules to an ingress fabric interface among the ingress fabric interfaces, and
the LL flow control circuitry is configured to issue LL flow control messages to link partners by communicating the LL flow control messages from an originating architecture component in the modularized packet routing architecture to an immediately preceding architecture component that is configured to perform queue stops and queue re-starts responsive to the LL flow control messages.
16. A method comprising:
in a network device:
communicating a link level (LL) flow control message between modularized link partners in a packet flow architecture in the network device, the LL flow control message being separate from data packets communicated by the device and the LL flow control message including a protocol frame that includes bits indicating congestion for each class of service (CoS); and
after sending the LL flow control message, alleviating blocking resulting from the LL flow control message by transmitting an end-to-end (EE) flow control message across the packet flow architecture from an egress module to a switching fabric and then the switching fabric multicasting the EE flow control message to a plurality of ingress fabric interfaces, wherein
each of the ingress fabric interfaces is a separate component from each of the egress fabric interfaces,
input data flows from an external source, through an ingress port, to an ingress fabric interface and then to the switching fabric,
output data flows from the switching fabric to an egress fabric interface and then, through an egress port, to an external destination,
the EE flow control message is transmitted directly from the egress module to the switching fabric without passing through any of a plurality of egress fabric interfaces, and
the communicating of the LL flow control message comprises executing LL flow control on input ports of the modularized link partners in the packet flow architecture.
17. The method of claim 16 , wherein executing LL flow control comprises:
executing LL flow control between physically connected link partners in the packet flow architecture.
18. The method of claim 16 , wherein effects of the LL flow control message precede effects of the EE flow control messages.