Resync transfer for recovering from storage site failure utilizing background scanner
Techniques are provided for performing a resync transfer to recover from a storage site failure. During normal operation of a first site hosting a first volume, data is replicated to a second volume hosted by a second site. If the first site fails, when clients are redirected to the second volume at the second site. When the first site recovers, data modifications made to the second volume are resynced back to the first volume. As part of synchronizing the first volume, a data warehouse is rebuilt at the first site in order to track the location of blocks present on the replication destination. Typically, the data modifications are transferred after the data warehouse is rebuilt, which results in significantly long resync times. The techniques provided herein decrease the resync time by either rebuilding the data warehouse in parallel with resyncing the data modifications or circumvent the need for rebuild.
1 . A method, comprising:
replicating data from a first volume hosted at a first site as a primary volume to a second volume hosted at a second site as a secondary volume;
in response to the first site experiencing a failure, promoting the second volume to be the primary volume and directing client I/O to the second volume for storing data modifications;
in response to the first site recovering, initiating a resync operation to transfer the data modifications from the second volume to the first volume for re-promoting the first volume to be the primary volume; and
during the resync operation:
logging database rebuild operations, for rebuilding a database file used to track blocks that are shared by files of at least one of the first volume or the second volume, into a log metafile that includes mappings between virtual volume block numbers of the second volume and virtual volume block numbers of the first volume; and
implementing a background scanner to reconcile information within the log metafile for rebuilding the database file, wherein the background scanner processes the mappings to perform block share operations to share file block numbers between the second volume and the first volume.
2 . The method of claim 1 , comprising:
representing a data warehouse, used to track locations of blocks within at least one of the first volume or the second volume, utilizing both the database file and the log metafile; and
utilizing both the database file and the log metafile to obtain a consistent state of at least one of the first volume or the second volume.
3 . The method of claim 1 , comprising:
evaluating the log metafile to determine whether a block has been processed as part of the resync operation.
4 . The method of claim 1 , comprising:
utilizing the log metafile and the database file to deduplicate blocks storing data of the first volume.
5 . The method of claim 1 , comprising:
in response to the resync operation completing, demoting the second volume to become the secondary volume and promoting the first volume to become the primary volume.
6 . The method of claim 1 , comprising:
maintaining a data warehouse for the secondary volume, wherein the data warehouse tracks blocks storing data of the secondary volume.
7 . The method of claim 1 , wherein virtual volume block number mappings are logged into the log metafile.
8 . The method of claim 1 , comprising:
populating a data warehouse maintained at the second site for the secondary volume with virtual volume block numbers used by the first volume.
9 . The method of claim 1 , comprising:
populating a data warehouse maintained at the second site for the secondary volume with mappings between virtual volume block numbers, used by the first volume for storing the data, to locations used by the second volume for storing corresponding data.
10 . A computing device comprising:
a memory storing instructions; and
a processor coupled to the memory, the processor configured to execute the instructions to perform operations comprising:
replicating data from a first volume hosted at a first site as a primary volume to a second volume hosted at a second site as a secondary volume;
in response to the first site experiencing a failure, promoting the second volume to be the primary volume and directing client I/O to the second volume for storing data modifications;
in response to the first site recovering, initiating a resync operation to transfer the data modifications from the second volume to the first volume for re-promoting the first volume to be the primary volume; and
during the resync operation:
logging database rebuild operations, for rebuilding a database file used to track blocks that are shared by files of at least one of the first volume or the second volume, into a log metafile that includes mappings between virtual volume block numbers of the second volume and virtual volume block numbers of the first volume; and
implementing a background scanner to reconcile information within the log metafile for rebuilding the database file, wherein the background scanner processes the mappings to perform block share operations to share file block numbers between the second volume and the first volume.
11 . The computing device of claim 10 , wherein the operations comprise:
logging virtual volume block numbers from the second site into the log metafile indexed by the virtual volume block numbers and mapped to virtual volume block numbers used by the first site.
12 . The computing device of claim 10 , wherein the operations comprise:
logging virtual volume block numbers from the second site into the log metafile indexed by the virtual volume block numbers, wherein the indexes are at fixed offsets and a maximum number of indices is derived from a largest virtual volume block number from the second site.
13 . The computing device of claim 10 , wherein the operations comprise:
specifying that the resync operation is complete in response to virtual volume block numbers from the second site being logged into the log metafile.
14 . The computing device of claim 10 , wherein the operations comprise:
asynchronously processing map entries within the log metafile, wherein for each map entry from the second volume to the first volume, a block share is performed between a file block number within a data warehouse for the second volume to a container file block number associated with the first volume.
15 . A non-transitory machine readable medium comprising instructions for performing a method, which when executed by a machine, causes the machine to perform operations comprising:
replicating data from a first volume hosted at a first site as a primary volume to a second volume hosted at a second site as a secondary volume;
in response to the first site experiencing a failure, promoting the second volume to be the primary volume and directing client I/O to the second volume for storing data modifications;
in response to the first site recovering, initiating a resync operation to transfer the data modifications from the second volume to the first volume for re-promoting the first volume to be the primary volume; and
during the resync operation:
logging database rebuild operations, for rebuilding a database file used to track blocks that are shared by files of at least one of the first volume or the second volume, into a log metafile that includes mappings between virtual volume block numbers of the second volume and virtual volume block numbers of the first volume; and
implementing a background scanner to reconcile information within the log metafile for rebuilding the database file, wherein the background scanner processes the mappings to perform block share operations to share file block numbers between the second volume and the first volume.
16 . The non-transitory machine readable medium of claim 15 , wherein the operations comprise:
asynchronously processing map entries within the log metafile, wherein for each map entry from the second volume to the first volume, a block share is performed between a file block number within a data warehouse for the second volume to a container file block number associated with the first volume.
17 . The non-transitory machine readable medium of claim 15 , wherein the operations comprise:
populating a data warehouse maintained at the second site for the secondary volume with virtual volume block numbers used by the first volume.
18 . The non-transitory machine readable medium of claim 15 , wherein the operations comprise:
populating a data warehouse maintained at the second site for the secondary volume with mappings between virtual volume block numbers, used by the first volume for storing the data, to locations used by the second volume for storing corresponding data.
19 . The non-transitory machine readable medium of claim 15 , wherein the operations comprise:
logging virtual volume block numbers from the second site into the log metafile indexed by the virtual volume block numbers and mapped to virtual volume block numbers used by the first site.
20 . The non-transitory machine readable medium of claim 15 , wherein the operations comprise:
logging virtual volume block numbers from the second site into the log metafile indexed by the virtual volume block numbers, wherein the indexes are at fixed offsets and a maximum number of indices is derived from a largest virtual volume block number from the second site.