Multi-tier mechanism to achieve high availability in a multi-controller system

1. A multi-controller system, comprising:

a processor configured to:

receive an indication that a network ping-based heartbeat mechanism has failed; and

in response to said indication, providing an alternative heartbeat at least in part by periodically updating a heartbeat data stored in an on-disk heartbeat data structure, in a portion of a disk that is available to both an active controller and a standby controller of the multi-controller system; and

a memory coupled to the processor and configured to provide the processor with instructions;

wherein the processor is further configured to detect that the on-disk heartbeat data structure-based alternative heartbeat has failed and in response switch over to a SCSI key refresh-based heartbeat mechanism.

2. The system of claim 1 , wherein the standby controller is configured to read the updated heartbeat data from the on-disk heartbeat data structure.

3. The system of claim 2 , wherein the standby controller is configured to read the heartbeat data from the on-disk heartbeat data structure; determine whether the heartbeat data read from the on-disk heartbeat data structure has been updated within a prescribed interval; and conclude based at least in part on the determination whether the active controller remains alive.

4. The system of claim 3 , wherein the standby controller is further configured to update a standby controller portion of the on-disk heartbeat data structure, based at least in part on a determination that the heartbeat data read by the standby controller from the on-disk heartbeat data structure has been updated within the prescribed interval.

5. The system of claim 4 , wherein the active controller is configured to read the standby controller portion of the on-disk heartbeat data structure; determine based at least in part on the data read from the whether the standby controller portion of the on-disk heartbeat data structure has been updated within a prescribed interval; and conclude based at least in part on the determination whether the standby controller is using the on-disk heartbeat data structure to determine whether the active controller is still alive.

6. The system of claim 1 , wherein the on-disk heartbeat data structure comprises an identical data structure stored on each of a plurality of storage disks comprising the multi-controller system.

7. The system of claim 1 , wherein the standby controller is configured to check to determine whether a key refresh has been performed by the active controller, and to determine based at least in part on a detected key refresh that the active controller is still alive.

8. The system of claim 7 , wherein the standby controller concludes upon detecting a first refreshed key, without checking one or more other keys, that the active controller is still alive.

9. The system of claim 1 , wherein the active controller is configured to refresh the respective SCSI key of a subset of disks comprising a storage array with which the multi-controller system is associated.

10. A method to achieve high availability in a multi-controller system, comprising:

receiving an indication that a network ping-based heartbeat mechanism has failed;

in response to said indication, providing an alternative heartbeat at least in part by periodically updating a heartbeat data stored in an on-disk heartbeat data structure, in a portion of a disk that is available to both an active controller and a standby controller of the multi-controller system; and

in response to detecting that the on-disk heartbeat data structure-based alternative heartbeat has failed, switching over to a SCSI key refresh-based heartbeat mechanism.

11. The method of claim 10 , wherein the standby controller is configured to read the updated heartbeat data from the on-disk heartbeat data structure.

12. The method of claim 11 , wherein the standby controller is configured to read the heartbeat data from the on-disk heartbeat data structure; determine whether the heartbeat data read from the on-disk heartbeat data structure has been updated within a prescribed interval; and conclude based at least in part on the determination whether the active controller remains alive.

13. The method of claim 12 , wherein the standby controller is further configured to update a standby controller portion of the on-disk heartbeat data structure, based at least in part on a determination that the heartbeat data read by the standby controller from the on-disk heartbeat data structure has been updated within the prescribed interval.

14. The method of claim 13 , wherein the active controller is configured to read the standby controller portion of the on-disk heartbeat data structure; determine based at least in part on the data read from the whether the standby controller portion of the on-disk heartbeat data structure has been updated within a prescribed interval; and conclude based at least in part on the determination whether the standby controller is using the on-disk heartbeat data structure to determine whether the active controller is still alive.

15. The method of claim 10 , wherein the on-disk heartbeat data structure comprises an identical data structure stored on each of a plurality of storage disks comprising the multi-controller system.

16. The method of claim 10 , wherein the standby controller is configured to check to determine whether a key refresh has been performed by the active controller, and to determine based at least in part on a detected key refresh that the active controller is still alive.

17. The method of claim 16 , wherein the standby controller concludes upon detecting a first refreshed key, without checking one or more other keys, that the active controller is still alive.

18. A computer program product to achieve high availability in a multi-controller system, the computer program product being embodied in a non-transitory computer readable medium and comprising computer instructions for:

receiving an indication that a network ping-based heartbeat mechanism has failed;

in response to said indication, providing an alternative heartbeat at least in part by periodically updating a heartbeat data stored in an on-disk heartbeat data structure, in a portion of a disk that is available to both an active controller and a standby controller of the multi-controller system; and

in response to detecting that the on-disk heartbeat data structure-based alternative heartbeat has failed, switching over to a SCSI key refresh-based heartbeat mechanism.

19. The computer program product of claim 18 , wherein the standby controller is configured to read the updated heartbeat data from the on-disk heartbeat data structure.

20. The computer program product of claim 19 , wherein the standby controller is configured to read the heartbeat data from the on-disk heartbeat data structure; determine whether the heartbeat data read from the on-disk heartbeat data structure has been updated within a prescribed interval; and conclude based at least in part on the determination whether the active controller remains alive.