Determine failed components in fault-tolerant memory
According to an example, a failed component in a fault-tolerant memory fabric may be determined by transmitting request packets along a plurality of routes between the redundancy controller and a media controller in periodic cycles. The redundancy controller may determine whether route failures for all of the plurality of routes have occurred within a number of consecutive periodic cycles. In response to determining that route failures for all of the plurality of routes have occurred within a number of consecutive periodic cycles, the media controller is established as failed. In response to determining that route failures for less than all of the plurality of routes have occurred within the number of consecutive periodic cycles, a fabric device is established as failed.
1. A method for determining failed component in a fault-tolerant memory fabric, the method comprising:
transmitting, by a redundancy controller from a plurality of redundancy controllers in the memory fabric, request packets along a plurality of routes between the redundancy controller and a media controller in periodic cycles;
determining whether route failures for all of the plurality of routes have occurred within a number of consecutive periodic cycles, wherein a route failure is determined to have occurred if a response packet to a request packet is not received along the same route on which the request packet was transmitted within a periodic cycle;
in response to determining that route failures for all of the plurality of routes have occurred within the number of consecutive periodic cycles, establishing that the media controller has failed; and
in response to determining that route failures for less than all of the plurality of routes have occurred within the number of consecutive periodic cycles, establishing that a fabric device has failed.
2. The method of claim 1 , wherein establishing that the media controller has failed comprises:
entering the redundancy controller into a degraded mode, wherein the degraded mode allows continued access to data previously stored on the failed media controller through use of redundant data stored on other media controllers.
3. The method of claim 1 , wherein establishing that the fabric device has failed comprises:
transmitting request packets along remaining routes that are functional; and
monitoring for route failures in the remaining functional routes.
4. The method of claim 3 , wherein monitoring for route failures in the remaining functional routes comprises:
shutting down the redundancy controller in response to a determination from the monitoring that route failures in the remaining functional routes have occurred during a periodic cycle subsequent to the number of consecutive periodic cycles.
5. The method of claim 4 , further comprising:
reenable the use of routes through the fabric devices that are dependent upon the redundancy controller after the memory fabric is repaired.
6. The method of claim 1 , wherein each of the plurality of redundancy controllers transmit request packets along a plurality of routes between each of the redundancy controllers and the media controller in periodic cycles, and each of the plurality of redundancy controllers determine whether route failures for all of the plurality of routes have occurred within a number of consecutive periodic cycles.
7. The method of claim 6 , further comprising:
in response to each of the plurality of redundancy controllers determining that route failures for all of the plurality of routes have occurred within the number of consecutive periodic cycles, transitioning each of the plurality of redundancy controllers to a degraded mode to prevent silent data corruption.
8. The method of claim 1 , wherein the request packets and the response packets are ping packets.
9. A redundancy controller to determine a failed component a fault-tolerant memory fabric, the redundancy controller comprising:
a packet module to send request ping-packets along a plurality of routes between the redundancy controller and a media controller in periodic cycles;
a determination module to determine whether route failures for all of the plurality of routes have occurred within a number of consecutive periodic cycles, wherein a route failure is determined to have occurred if a ping-response packet to a ping-request packet is not received along the same route on which the request packet was transmitted within a periodic cycle; and
a designation module to
designate a failed media controller in response to determining that route failures for all of the plurality of routes have occurred within the number of consecutive periodic cycles, and
designate a failed fabric device in response to determining that route failures for less than all of the plurality of routes have occurred within the number of consecutive periodic cycles.
10. The redundancy controller of claim 9 , wherein to designate a failed media controller, the designation module is to:
enter a degraded mode, wherein the degraded mode allows continued access to data previously stored on the failed media controller through use of redundant data stored on other media controllers.
11. The redundancy controller of claim 9 , wherein to designate a failed fabric device, the designation module is to:
transmit request ping-packets along the remaining routes that are functional; and
monitor for route failures in the remaining functional routes.
12. The redundancy controller of claim 11 , wherein to monitor for route failures in the remaining functional routes, designation module is to:
shut down the redundancy controller in response to a determination from the monitoring that route failures in the remaining functional routes have occurred during a periodic cycle subsequent to the number of consecutive periodic cycles; and
reenable the use of routes through fabric devices that are dependent upon the redundancy controller after the memory fabric is repaired.
13. A non-transitory computer readable medium to determine a failed component in a fault-tolerant memory fabric, including machine readable instructions executable by a processor to:
transmit request packets along a plurality of routes between a redundancy controller and a media controller in periodic cycles;
determine whether route failures for all of the plurality of routes have occurred within a number of consecutive periodic cycles, wherein a route failure is determined to have occurred if a response packet to a request packet is not received along the same route on which the request packet was transmitted within a periodic cycle;
in response to determining that route failures for all of the plurality of routes have occurred within the number of consecutive periodic cycles,
establish that the media controller has failed, and
activate a degraded mode;
in response to determining that route failures for less than all of the plurality of routes have occurred within the number of consecutive periodic cycles,
establish that a fabric device has failed, and
monitor for route failures in remaining functional routes.
14. The non-transitory computer readable medium of claim 13 , wherein to monitor for route failures in the remaining functional routes, the machine readable instructions are executable by the processor to:
shut down the redundancy controller in response to a determination from the monitoring that route failures in the remaining functional routes have occurred during a periodic cycle subsequent to the number of consecutive periodic cycles.
15. The non-transitory computer readable medium of claim 13 , wherein in response the redundancy controller activating the degraded mode, the machine readable instructions are executable by the processor to prevent silent data corruption in the fault-tolerant memory.