Implementing automatic switchover
One or more techniques and/or computing devices are provided for automatic switchover implementation. For example, a first storage controller, of a first storage cluster, may have a disaster recovery relationship with a second storage controller of a second storage cluster. In the event the first storage controller fails, the second storage controller may automatically switchover operation from the first storage controller to the second storage controller for providing clients with failover access to data previously accessible to the clients through the first storage controller. The second storage controller may detect, cross-cluster, a failure of the first storage controller utilizing remote direct memory access (RDMA) read operations to access heartbeat information, heartbeat information stored within a disk mailbox, and/or service processor traps. In this way, the second storage controller may efficiently detect failure of the first storage controller to trigger automatic switchover for non-disruptive client access to data.
1. A method comprising:
determining that a memory section is designated for heartbeat information exchange from a first node to a second node;
performing a remote direct memory access read operation to access the memory section for obtaining a current heartbeat status of the first node;
determining a write caching synchronization state between the first node and the second node; and
initiating an automatic switchover operation from the first node to the second node based upon the current heartbeat status indicating a failure of the first node and the write caching synchronization state indicating a synchronous state.
2. The method of claim 1 , wherein the current heartbeat status specifies a storage controller reboot as the failure.
3. The method of claim 1 , wherein the current heartbeat status specifies a state transition of the first node.
4. The method of claim 1 , wherein the heartbeat information exchange corresponds to a series of sequence numbers used to indicate progress of the first node.
5. The method of claim 1 , wherein the initiating an automatic switchover operation comprises:
evaluating responsiveness to a communication signal sent from the second node to the first node to determine whether the failure is a false trigger.
6. The method of claim 1 , comprising:
initiating a manual switchover operation based upon a determination that storage and a main controller of a cluster comprising the first node are unavailable.
7. The method of claim 1 , comprising:
refraining from initiating the automatic switchover operation based upon the write caching synchronization state indicating a non-synchronous state.
8. The method of claim 1 , comprising:
reading the write caching synchronization state from a first disk mailbox of the first node.
9. The method of claim 1 , wherein the first node is configured according to a single controller cluster configuration and the second node is configured according to the single controller cluster configuration.
10. The method of claim 1 , wherein the initiating an automatic switchover operation comprises:
initiating the automatic switchover operation based upon both the current heartbeat status and a second current heartbeat status, read from a first disk mailbox, indicating the failure.
11. The method of claim 10 , comprising:
determining the failure as a power loss failure based upon both the current heartbeat status and the second current heartbeat status indicating the failure.
12. The method of claim 10 , comprising:
initiating the automatic switchover operation after a threshold timeout based upon both the current heartbeat status and the second current heartbeat status indicating the failure.
13. A non-transitory machine readable medium comprising machine executable code which when executed by at least one machine, causes the machine to:
determine that a memory section is designated for heartbeat information exchange from a first node to a second node;
perform a remote direct memory access read operation to access the memory section for obtaining a current heartbeat status of the first node;
determine a write caching synchronization state between the first node and the second node; and
initiate an automatic switchover operation from the first node to the second node based upon the current heartbeat status indicating a failure of the first node and the write caching synchronization state indicating a synchronous state.
14. The non-transitory machine readable medium of claim 13 , wherein the machine executable code causes the machine to:
initiate the automatic switchover operation based upon both the current heartbeat status and a second current heartbeat status, read from a first disk mailbox, indicating the failure.
15. The non-transitory machine readable medium of claim 13 , wherein the machine executable code causes the machine to:
initiate the automatic switchover operation after a timeout.
16. The non-transitory machine readable medium of claim 13 , wherein the machine executable code causes the machine to:
evaluate responsiveness to a communication signal sent from the second node to the first node to determine whether the failure is a false trigger.
17. The non-transitory machine readable medium of claim 13 , wherein the machine executable code causes the machine to:
initiate the automatic switchover operation based upon storage and a main controller of a cluster comprising the first node being available.
18. The non-transitory machine readable medium of claim 13 , wherein the initiating an automatic switchover operation comprises:
refrain from initiating the automatic switchover operation based upon the write caching synchronization state indicating a non-synchronous state.
19. A computing device comprising:
a memory comprising machine executable code; and
a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to:
determine that a memory section is designated for heartbeat information exchange from a first node to a second node;
perform a remote direct memory access read operation to access the memory section for obtaining a current heartbeat status of the first node;
determine a write caching synchronization state between the first node and the second node; and
initiate an automatic switchover operation from the first node to the second node based upon the current heartbeat status indicating a failure of the first node and the write caching synchronization state indicating a synchronous state.
20. The computing device of claim 19 , wherein the machine executable code causes the processor to:
refrain from initiating the automatic switchover operation based upon the write caching synchronization state indicating a non-synchronous state.