Multi-stage switch fabric fault detection and handling
In some examples, a switching system includes a plurality of fabric endpoints and a multi-stage switching fabric having a plurality of fabric planes each having a plurality of stages to switch data units between any of the plurality of fabric endpoints. A fabric endpoint of the fabric endpoints is configured to send, to a switch of a first one of the stages and within a first fabric plane of the plurality of fabric planes, a self-ping message destined for the fabric endpoint. The fabric endpoint is configured to send, in response to determining the fabric endpoint has not received the self-ping message after a predetermined time, an indication of a connectivity fault for the first fabric plane.
1. A switching system comprising:
a plurality of fabric endpoints;
a multi-stage switching fabric having a plurality of fabric planes each having a plurality of stages to switch data units between any of a plurality of fabric endpoints,
wherein a first-stage switch of a first fabric plane of the plurality of fabric planes is configured to receive, from an intermediate-stage switch or first final-stage switch of the first fabric plane, an indication of a link error for a failed link connecting a first switch of the first fabric plane to a second switch of the first fabric plane;
wherein the first-stage switch is configured to send, to a second final-stage switch of the first fabric plane, a first fabric link fault signaling cell that identifies the failed link, and
wherein the first-stage switch is configured to send, to a third final-stage switch of the first fabric plane, a second fault fabric link fault signaling cell that identifies the failed link.
2. The switching system of claim 1 ,
wherein the first switch of the first fabric plane comprises the intermediate-stage switch, and
wherein the second switch of the first fabric plane comprises the first final-stage switch.
3. The switching system of claim 1 ,
wherein the first switch of the first fabric plane comprises the first-stage switch, and
wherein the second switch of the first fabric plane comprises the intermediate-stage switch.
4. The switching system of claim 1 ,
wherein the first final-stage switch is coupled to the first-stage switch by a direct communication link, and
wherein the first-stage switch receives the indication of the link error for the failed link via the direct communication link.
5. The switching system of claim 1 , wherein the intermediate-stage switch disables spray on the failed link.
6. The switching system of claim 1 , wherein the first-stage switch is configured to disable spray on the failed link in response to receiving the indication of the link error for the failed link.
7. The switching system of claim 1 , where the second final-stage switch is configured to send, to at least one of the plurality of fabric endpoints, a fault notification cell comprising the indication of the link error for the failed link.
8. The switching system of claim 1 , wherein the first fabric link fault signaling cell comprises an address for the second final-stage switch of the first fabric plane.
9. The switching system of claim 1 , wherein the first fabric link fault signaling cell has a first-stage to final-stage (F 1 _TO_F 3 ) cell type.
10. The switching system of claim 1 ,
wherein one of the plurality of endpoints is configured to send a third fabric link fault signaling cell comprising the indication of the link error for the failed link.
11. A method of verifying fabric connectivity with a multi-stage switch fabric having a plurality of fabric planes each having a plurality of stages to switch data units between any of a plurality of fabric endpoints, the method comprising:
receiving, by a first-stage switch of a first fabric plane of the plurality of fabric planes, from an intermediate-stage switch or first final-stage switch of the first fabric plane, an indication of a link error for a failed link connecting a first switch of the first fabric plane to a second switch of the first fabric plane;
sending, by the first-stage switch, to a second final-stage switch of the first fabric plane, a first fabric link fault signaling cell that identifies the failed link, and
sending, by the first-stage switch, to a third final-stage switch of the first fabric plane, a second fault fabric link fault signaling cell that identifies the failed link.
12. The method of claim 11 ,
wherein the first switch of the first fabric plane comprises the intermediate-stage switch, and
wherein the second switch of the first fabric plane comprises the first final-stage switch.
13. The method of claim 11 ,
wherein the first switch of the first fabric plane comprises the first-stage switch, and
wherein the second switch of the first fabric plane comprises the intermediate-stage switch.
14. The method of claim 11 ,
wherein the first final-stage switch is coupled to the first-stage switch by a direct communication link, and
wherein the first-stage switch receives the indication of the link error for the failed link via the direct communication link.
15. The method of claim 11 , further comprising:
disabling, in response to receiving the first fabric link fault signaling cell, spray on the failed link.
16. The method of claim 11 , further comprising:
disabling, by the first-stage switch, in response to the indication of the link error for the failed link, spray on the failed link.
17. The method of claim 11 , further comprising:
sending, by the second final-stage switch, to at least one of the plurality of fabric endpoints, a fault notification cell comprising the indication of the link error for the failed link.
18. The method of claim 11 , wherein the first fabric link fault signaling cell comprises an address for the second final-stage switch of the first fabric plane.
19. The method of claim 11 , further comprising:
sending, by one of the plurality of endpoints, a third fabric link fault signaling cell comprising the indication of the link error for the failed link.
20. A non-transitory computer-readable medium comprising instructions for causing a multi-stage switch fabric, having a plurality of fabric planes each having a plurality of stages to switch data units between any of a plurality of fabric endpoints, to perform operations comprising:
receiving, by a first-stage switch of a first fabric plane of the plurality of fabric planes, from an intermediate-stage switch or first final-stage switch of the first fabric plane, an indication of a link error for a failed link connecting a first switch of the first fabric plane to a second switch of the first fabric plane;
sending, by the first-stage switch, to a second final-stage switch of the first fabric plane, a first fabric link fault signaling cell that identifies the failed link, and
sending, by the first-stage switch, to a third final-stage switch of the first fabric plane, a second fault fabric link fault signaling cell that identifies the failed link.