On-demand serverless disaster recovery
Techniques are provided for on-demand serverless disaster recovery. A primary node may host a primary volume. Snapshots of the primary volume may be backed up to an object store. In response to failure, a secondary node and/or an on-demand volume may be created on-demand. The secondary node may provide clients with failover access to the on-demand volume while a restore process restores a snapshot of the primary volume to the on-demand volume. In some embodiments, there was no secondary node and/or on-demand volume while the primary node was operational. This conserves computing resources that would be wasted by otherwise hosting the secondary node and/or on-demand volume while clients were able to access the primary volume through the primary node. Modifications directed to the on-demand volume are incrementally backed up to the object store for subsequently restoring the primary volume after recovery.
1. A method comprising:
in response to determining that a primary volume hosted by a primary node has experienced a failure, creating a secondary node on-demand in response to the failure;
generating an on-demand volume accessible to the secondary node, wherein a restore process is initiated to restore the on-demand volume to a state of the primary volume captured by a snapshot stored within an object store; and
in response to copying metadata in the snapshot to the secondary node, providing clients with read and write access to the on-demand volume through the secondary node during the restore process.
2. The method of claim 1 , wherein the providing the clients with read and write access comprises:
retrieving a block of data from the snapshot in the object store based upon a request from a client for the block of data not yet restored to the on-demand volume.
3. The method of claim 1 , wherein the providing the clients with read and write access comprises:
receiving a request from a client for a block of data not yet restored from the snapshot to the on-demand volume
retrieving the block from the snapshot in the object store; and
caching the block within a cloud block map as a cached block accessible to the client.
4. The method of claim 1 , wherein the providing the clients with read and write access comprises:
retrieving a block of data from the snapshot in the object store based upon a request from a client for the block of data not yet restored to the on-demand volume;
caching the block within a cloud block map as a cached block accessible to the client, wherein the cloud block map maps a cloud block number of the block to a physical volume block number of the cached block.
5. The method of claim 1 , comprising:
incrementally backing up modifications made to the on-demand volume as on-demand volume snapshots stored within the object store; and
evaluating a cloud block map, used to cache blocks not yet restored by the restore process, to identify unmodified restored data within the on-demand volume and modified data corresponding to the modifications to incrementally back up to the object store.
6. The method of claim 1 , comprising:
configuring a serverless disaster recovery relationship for the primary node.
7. The method of claim 1 , comprising:
configuring a serverless disaster recovery relationship for the primary node; and
generating the secondary node on-demand in response to the failure.
8. The method of claim 1 , wherein the metadata comprises root metadata of a file system.
9. A computing device comprising:
a memory comprising machine executable code; and
a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to:
create a secondary node on-demand to takeover for a primary node;
generate an on-demand volume accessible to the secondary node, wherein a restore process is initiated to restore the on-demand volume to a state of a primary volume captured by a snapshot stored within an object store; and
in response to copying metadata in the snapshot to the secondary node, providing clients with read and write access to the on-demand volume through the secondary node during the restore process.
10. The computing device of claim 9 , wherein the machine executable code causes the processor to:
retrieve a block of data from the snapshot in the object store based upon a request from a client for the block of data not yet restored to the on-demand volume.
11. The computing device of claim 9 , wherein the machine executable code causes the processor to:
receive a request from a client for a block of data not yet restored from the snapshot to the on-demand volume
retrieve the block from the snapshot in the object store; and
cache the block within a cloud block map as a cached block accessible to the client.
12. The computing device of claim 9 , wherein the machine executable code causes the processor to:
retrieve a block of data from the snapshot in the object store based upon a request from a client for the block of data not yet restored to the on-demand volume;
cache the block within a cloud block map as a cached block accessible to the client, wherein the cloud block map maps a cloud block number of the block to a physical volume block number of the cached block.
13. The computing device of claim 9 , wherein the machine executable code causes the processor to:
incrementally back up modifications made to the on-demand volume as on-demand volume snapshots stored within the object store; and
evaluate a cloud block map, used to cache blocks not yet restored by the restore process, to identify unmodified restored data within the on-demand volume and modified data corresponding to the modifications to incrementally back up to the object store.
14. The computing device of claim 9 , wherein the machine executable code causes the processor to:
incrementally backing up modifications made to the on-demand volume as on-demand volume snapshots stored within the object store; and
in response to the primary volume recovering from a failure, incrementally restore the modifications from the on-demand volume snapshots to the primary volume.
15. A non-transitory machine readable medium comprising instructions for performing a method, which when executed by a machine, causes the machine to perform operations comprising:
in response to determining that a primary volume hosted by a primary node has experienced a failure, creating a secondary node on-demand in response to the failure;
generating an on-demand volume accessible to the secondary node, wherein a restore process is initiated to restore the on-demand volume to a state of the primary volume captured by a snapshot stored within an object store; and
in response to copying metadata in the snapshot to the secondary node, providing clients with read and write access to the on-demand volume through the secondary node during the restore process.
16. The non-transitory machine readable medium of claim 15 , wherein the operations comprise:
retrieving a block of data from the snapshot in the object store based upon a request from a client for the block of data not yet restored to the on-demand volume.
17. The non-transitory machine readable medium of claim 15 , wherein the operations comprise:
receiving a request from a client for a block of data not yet restored from the snapshot to the on-demand volume retrieving the block from the snapshot in the object store; and
caching the block within a cloud block map as a cached block accessible to the client.
18. The non-transitory machine readable medium of claim 15 , wherein the operations comprise:
retrieving a block of data from the snapshot in the object store based upon a request from a client for the block of data not yet restored to the on-demand volume;
caching the block within a cloud block map as a cached block accessible to the client, wherein the cloud block map maps a cloud block number of the block to a physical volume block number of the cached block.
19. The non-transitory machine readable medium of claim 15 , wherein the operations comprise:
incrementally backing up modifications made to the on-demand volume as on-demand volume snapshots stored within the object store; and
evaluating a cloud block map, used to cache blocks not yet restored by the restore process, to identify unmodified restored data within the on-demand volume and modified data corresponding to the modifications to incrementally back up to the object store.
20. The non-transitory machine readable medium of claim 15 , wherein the operations comprise:
incrementally backing up modifications made to the on-demand volume as on-demand volume snapshots stored within the object store; and
in response to the primary volume recovering from the failure, incrementally restoring the modifications from the on-demand volume snapshots to the primary volume.