On-demand restore of a snapshot to an on-demand volume accessible to clients
Techniques are provided for caching data during an on-demand restore using a cloud block map. A client may be provided with access to an on-demand volume during a restore process that copies backup data from a snapshot within a remote object store to the on-demand volume stored within local storage. In response to receiving a request from the client for a block of the backup data not yet restored from the snapshot to the on-demand volume, the block may be retrieved from the snapshot in the remote object store. The block may be cached within a cloud block map stored within the local storage as a cached block. The client may be provided with access to the cached block.
1. A method comprising:
maintaining metadata comprising cloud block numbers identifying blocks comprising backup data of a snapshot stored within objects in a remote object store, wherein a cloud block number of a block comprises a sequence number of an object and a slot number of a slot within the object in which backup data of the block is stored; and
providing a client with access to an on-demand volume during a restore process that copies the backup data from the snapshot within the remote object store to the on-demand volume, comprising:
in response to receiving a request from the client through the on-demand volume for the block, on-demand retrieving from the snapshot and caching the block for client access based upon the metadata mapping the cloud block number of the block to an indication that the block has not yet been restored; and
implementing the restore process to copy the backup data from the snapshot within the remote object store to the on-demand volume using the cloud block numbers to identify and retrieve the backup data from the objects.
2. The method of claim 1 , comprising:
initializing the metadata with the cloud block numbers mapped to absent indicators indicating that the blocks have not yet been restored from the snapshot in the object store.
3. The method of claim 1 , wherein the restore process uses the sequence number of the cloud block number to identify the object and uses the slot number of the cloud block number to identify and retrieve the backup data of the block from the object for restoring the backup data of the block into the on-demand volume.
4. The method of claim 1 , comprising:
in response to the metadata indicating that the block has been restored to the on-demand volume based upon the metadata mapping a virtual volume block number for the block to a physical volume block number for the block, retrieving the block from local storage for client access.
5. The method of claim 1 , wherein the providing the client with access to the on-demand volume comprises:
in response to the metadata indicating that the block has been restored to the on-demand volume, retrieving the block from local storage; and
providing the block to the client in response to the request.
6. The method of claim 5 , comprising:
determining that the block has been restored based upon the metadata comprising a virtual volume block number for the block.
7. The method of claim 6 , wherein the virtual volume block number is mapped to a physical volume block number used to retrieve the block from local storage.
8. The method of claim 1 , comprising:
in response to the restore process completing, converting the on-demand volume to a volume exposed to clients for read and write access of data through the volume.
9. The method of claim 1 , comprising:
in response to the restore process restoring the blocks as restored blocks within local storage, overwriting cloud block numbers within the metadata for the blocks with virtual volume block numbers for the restored blocks and overwriting absent indicators within the metadata for the blocks with physical volume block numbers for the restored blocks.
10. The method of claim 1 , comprising:
in response to on-demand retrieving and caching the block as a cached block, retaining the cloud block number within the metadata for the block and replacing the absent indicator within the metadata for the block with a physical volume block number of the cached block.
11. The method of claim 1 , comprising:
in response to the restore process restoring a block from the snapshot to the on-demand volume, replacing a cloud block number of the block with a virtual volume block number for the block.
12. The method of claim 11 , comprising:
replacing an absent indicator mapped to the cloud block number with a physical volume block number mapped to the virtual volume block number.
13. A computing device comprising:
a memory comprising machine executable code; and
a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to:
maintain metadata comprising cloud block numbers identifying blocks comprising backup data of a snapshot stored within objects in a remote object store, wherein a cloud block number of a block comprises a sequence number of an object and a slot number of a slot within the object in which backup data of the block is stored; and
provide a client with access to an on-demand volume during a restore process that copies the backup data from the snapshot within the remote object store to the on-demand volume, comprising:
in response to receiving a request from the client through the on-demand volume for the block, on-demand retrieve from the snapshot and caching the block for client access based upon the metadata mapping the cloud block number of the block to an indication that the block has not yet been restored; and
implement the restore process to copy the backup data from the snapshot within the remote object store to the on-demand volume using the cloud block numbers to identify and retrieve the backup data from the objects.
14. The computing device of claim 13 , wherein the machine executable code causes the processor to:
populate the metadata with virtual volume block number entries mapped to physical volume block number entries.
15. The computing device of claim 14 , wherein a virtual volume block number entry is populated with a cloud block number and a corresponding physical volume block number entry is populated with an absent indicator for a block not yet restored from the snapshot.
16. The computing device of claim 14 , wherein a virtual volume block number entry is populated with a virtual volume block number and a corresponding physical volume block number entry is populated with a physical volume block number for a block restored from the snapshot to the on-demand volume.
17. The computing device of claim 13 , wherein the machine executable code causes the processor to:
in response to receiving a request for a block from the client and the metadata indicating that the block has not yet been restored to the on-demand volume, retrieve the block from the snapshot in the remote object store to provide to the client in response to the request.
18. The computing device of claim 13 , wherein the machine executable code causes the processor to:
in response to receiving a request for a block from the client and the metadata indicating that the block has been restored to the on-demand volume, retrieve the block from local storage to provide to the client in response to the request.
19. A non-transitory machine readable medium comprising instructions for performing a method, which when executed by a machine, causes the machine to:
maintain metadata identifying blocks comprising backup data of a snapshot stored within objects in a remote object store; and
provide a client with access to an on-demand volume during a restore process that copies the backup data from the snapshot within the remote object store to the on-demand volume, comprising:
in response to receiving a request from the client through the on-demand volume for the block, on-demand retrieve from the snapshot and caching the block for client access based upon the metadata mapping the block to an indication that the block has not yet been restored; and
implement the restore process to copy the backup data from the snapshot within the remote object store to the on-demand volume using the metadata to identify and retrieve the backup data from the objects.
20. The non-transitory machine readable medium of claim 19 , wherein the instructions cause the machine to:
in response to the restore process restoring a block from the snapshot to the on-demand volume, replace, within the metadata, a cloud block number of the block with a virtual volume block number for the block.