Orchestrating high availability failover for virtual machines stored on distributed object-based storage

1. A method for orchestrating high availability (HA) failover for virtual machines (VMs) running on host systems of a host cluster, wherein the host cluster aggregates locally-attached storage resources of the host systems to provide an object store, and wherein persistent data for one or more VMs is stored as per-VM storage objects across the locally-attached storage resources comprising the object store, the method comprising:

determining, by a host system in the host cluster executing a HA module, a VM to be restarted on an active host system in the host cluster;

determining, by the host system, if the VM's persistent data is stored in the object store;

if the VM's persistent data is not stored in the object store, adding, by the host system, the VM to a list of VMs to be immediately restarted; and

if the VM's persistent data is stored in the object store, checking, by the host system, if the VM is accessible to the host system by querying a storage layer of the host system configured to manage the object store.

2. The method of claim 1 wherein the VM is accessible to the host system if all of the VM's storage objects in the object store are accessible to the host system.

3. The method of claim 2 wherein the HA module is a master HA module.

4. The method of claim 1 further comprising:

if the VM is accessible to the host system, adding the VM to the list of VMs to be immediately restarted; and

if the VM is not accessible to the host system, adding the VM to a list of VMs waiting for an accessibility change.

5. The method of claim 4 further comprising processing the list of VMs to be immediately restarted, the processing comprising, for each VM in the list:

placing and restarting the VM on an active host system; and

if the restarting of the VM is unsuccessful:

re-checking if the VM is accessible to the host system; and

if the VM is not accessible to the host system:

removing the VM from the list of VMs to be immediately restarted; and

adding the VM to the list of VMs waiting for an accessibility change.

6. The method of claim 4 further comprising processing the list of VMs waiting for an accessibility change, the processing comprising, for each VM in the list:

determining, after a period of time, that the VM has become accessible to the host system;

removing the VM from the list of VMs waiting for an accessibility change; and

adding the VM to the list of VMs to be immediately restarted.

7. The method of claim 6 wherein the host system determines that the VM has become accessible by receiving one or more notifications from the storage layer, and wherein the one or more notifications are generated by the storage layer as VMs become accessible in response to a resolution event.

8. A non-transitory computer readable storage medium having stored thereon computer software executable by a processor of a host system, the computer software embodying a method for orchestrating high availability (HA) failover for virtual machines (VMs) running on the host system and other host systems of a host cluster, wherein the host cluster aggregates locally-attached storage resources of the host systems to provide an object store, and wherein persistent data for one or more VMs is stored as per-VM storage objects across the locally-attached storage resources comprising the object store, the method comprising:

determining, via a HA module executing on the host system, a VM to be restarted on an active host system in the host cluster;

determining, via the HA module, if the VM's persistent data is stored in the object store;

if the VM's persistent data is not stored in the object store, adding the VM to a list of VMs to be immediately restarted; and

if the VM's persistent data is stored in the object store, checking if the VM is accessible to the host system by querying a storage layer of the host system configured to manage the object store.

9. The non-transitory computer readable storage medium of claim 8 wherein the VM is accessible to the host system if all of the VM's storage objects in the object store are accessible to the host system.

10. The non-transitory computer readable storage medium of claim 9 wherein the HA module is a master HA module.

11. The non-transitory computer readable storage medium of claim 8 wherein the method further comprises:

if the VM is accessible to the host system, adding the VM to the list of VMs to be immediately restarted; and

if the VM is not accessible to the host system, adding the VM to a list of VMs waiting for an accessibility change.

12. The non-transitory computer readable storage medium of claim 11 wherein the method further comprises processing the list of VMs to be immediately restarted, the processing comprising, for each VM in the list:

placing and restarting the VM on an active host system; and

if the restarting of the VM is unsuccessful:

re-checking if the VM is accessible to the host system; and

if the VM is not accessible to the host system:

removing the VM from the list of VMs to be immediately restarted; and

adding the VM to the list of VMs waiting for an accessibility change.

13. The non-transitory computer readable storage medium of claim 11 wherein the method further comprises processing the list of VMs waiting for an accessibility change, the processing comprising, for each VM in the list:

determining, after a period of time, that the VM has become accessible to the host system;

removing the VM from the list of VMs waiting for an accessibility change; and

adding the VM to the list of VMs to be immediately restarted.

14. The non-transitory computer readable storage medium of claim 13 wherein the host system determines that the VM has become accessible by receiving one or more notifications from the storage layer of the host cluster, and wherein the one or more notifications are generated by the storage layer as VMs become accessible in response to a resolution event.

15. A host system comprising:

a processor; and

a non-transitory computer readable medium having stored thereon instructions that, when executed by the processor, causes the processor to perform a method for orchestrating high availability (HA) failover for virtual machines (VMs) running on the host system and other host systems of a host cluster, wherein the host cluster aggregates locally-attached storage resources of the host systems to provide an object store, and wherein persistent data for one or more VMs is stored as per-VM storage objects across the locally-attached storage resources comprising the object store, the method comprising:

determining, via a HA module executing on the host system, a VM to be restarted on an active host system in the host cluster;

determining, via the HA module, if the VM's persistent data is stored in the object store;

if the VM's persistent data is not stored in the object store, adding the VM to a list of VMs to be immediately restarted; and

if the VM's persistent data is stored in the object store, checking if the VM is accessible to the host system by querying a storage layer of the host system configured to manage the object store.

16. The host system of claim 15 wherein the VM is accessible to the host system if all of the VM's storage objects in the object store are accessible to the host system.

17. The host system of claim 16 wherein the HA module is a master HA module.

18. The host system of claim 15 wherein the method further comprises:

if the VM is accessible to the host system, adding the VM to the list of VMs to be immediately restarted; and

if the VM is not accessible to the host system, adding the VM to a list of VMs waiting for an accessibility change.

19. The host system of claim 18 wherein the method further comprises processing the list of VMs to be immediately restarted, the processing comprising, for each VM in the list:

placing and restarting the VM on an active host system; and

if the restarting of the VM is unsuccessful:

re-checking if the VM is accessible to the host system; and

if the VM is not accessible to the host system:

removing the VM from the list of VMs to be immediately restarted; and

adding the VM to the list of VMs waiting for an accessibility change.

20. The host system of claim 18 wherein the method further comprises processing the list of VMs waiting for an accessibility change, the processing comprising, for each VM in the list:

determining, after a period of time, that the VM has become accessible to the host system;

removing the VM from the list of VMs waiting for an accessibility change; and

adding the VM to the list of VMs to be immediately restarted.

21. The host system of claim 20 wherein the host system determines that the VM has become accessible by receiving one or more notifications from the storage layer, and wherein the one or more notifications are generated by the storage layer as VMs become accessible in response to a resolution event.