Full sweep disk synchronization in a storage system
Described embodiments provide systems and methods for synchronizing a production volume and a backup volume of a storage system. A first thin volume is created and associated with the production volume. A first replica of the production volume is generated by copying data from the production volume to a replica volume. During the copying, an I/O request to be written to the production volume may be received. Data from the I/O request is written to the first thin volume and data changed due to the I/O request is tracked in metadata associated with the production volume and the first thin volume. A size of the first thin volume is checked, and when the size of the first thin volume is below a threshold, changes from the first thin volume are applied asynchronously to the backup storage.
1. A method comprising:
creating a first thin volume associated with a production volume;
generating a first replica of the production volume by copying data from the production volume to a replica volume;
during the copying:
receiving an I/O request to be written to the production volume;
writing data from the I/O request to the first thin volume and tracking data changed due to the I/O request in metadata associated with the production volume and the first thin volume;
checking data size of the first thin volume;
when the size of the first thin volume is below a threshold, asynchronously applying changes from the first thin volume to a backup storage;
when the size of the first thin volume is at or above a threshold;
determining whether a region of the production volume associated with the received I/O request has already been copied from the production volume to the first thin volume; and
when the region is already copied, overwriting an associated region in the first thin volume, whereby an amount of metadata to track changed data is reduced.
2. The method of claim 1 , further comprising performing the copying as a background process while processing new received I/O requests.
3. The method of claim 1 , further comprising, when the size of the first thin volume is below the threshold and the region is already copied from the production volume to the replica volume:
when the copying data from the production volume to the replica volume is complete, generating a second thin volume and writing data for new received I/O requests to the second thin volume.
4. The method of claim 3 , further comprising, when the size of the first thin volume is at or above the threshold and the region is already copied from the production volume to the replica volume:
copying data from the first thin volume to the replica volume;
during the copying from the first thin volume to the replica volume, tracking data changed due to a new I/O request in metadata associated with the first thin volume; and
when the copying from the first thin volume to the replica volume is complete, setting, based upon the metadata, locations as dirty data to be copied from the production volume to the replica volume.
5. The method of claim 3 , wherein upon determining the data size of the first thin volume meets or exceeds the threshold while data is being copied from the production volume to the replica volume:
creating a third thin volume having corresponding metadata;
upon receiving a next I/O request, determining if an address associated with the next I/O request is already written to the replica volume;
if the address is already copied to the replica volume, writing the data of the I/O request to the replica volume; and
if the address has not been copied from the production volume to the replica volume, updating the corresponding metadata.
6. The method of claim 1 , further comprising, when the region of the production volume associated with the received I/O request has not yet been copied from the production volume to the replica volume, tracking data changed due to the I/O request in metadata associated with the first thin volume.
7. A system comprising:
a processor; and
memory storing computer program code that when executed on the processor causes the processor to operate a storage system, the storage system operable to perform the operations of:
creating a first thin volume associated with a production volume;
generating a first replica of the production volume by copying data from the production volume to a replica volume;
during the copying:
receiving an I/O request to be written to the production volume;
writing data from the I/O request to the first thin volume and tracking data changed due to the I/O request in metadata associated with the production volume and the first thin volume;
checking data size of the first thin volume;
when the size of the first thin volume is below a threshold, asynchronously applying changes from the first thin volume to a backup storage;
when the size of the first thin volume is at or above a threshold, the storage system is operable to perform the operations of:
determining whether a region of the production volume associated with the received I/O request has already been copied from the production volume to the first thin volume; and
when the region is already copied, overwriting an associated region in the first thin volume, whereby an amount of metadata to track changed data is reduced.
8. The system of claim 7 , wherein the storage system is operable to perform the operation of performing the copying as a background process while processing new received I/O requests.
9. The system of claim 7 , wherein, when the size of the first thin volume is below the threshold and when the region is already copied from the production volume to the replica volume, the storage system is operable to perform the operation of:
when the copying data from the production volume to the replica volume is complete, generating a second thin volume and writing data for new received I/O requests to the second thin volume.
10. The system of claim 9 , wherein, when the size of the first thin volume is at or above the threshold and the region is already copied from the production volume to the replica volume, the storage system is operable to perform the operations of:
copying data from the first thin volume to the replica volume;
during the copying from the first thin volume to the replica volume, tracking data changed due to a new I/O request in metadata associated with the first thin volume; and
when the copying from the first thin volume to the replica volume is complete, setting, based upon the metadata, locations as dirty data to be copied from the production volume to the replica volume.
11. The system of claim 7 , wherein, when the region of the production volume associated with the received I/O request has not yet been copied from the production volume to the replica volume, the storage system is operable to perform the operation of tracking data changed due to the I/O request in metadata associated with the first thin volume.
12. A computer program product including a non-transitory computer readable storage medium having computer program code encoded thereon that when executed on a processor of a computer causes the computer to operate a storage system, the computer program product comprising:
computer program code for creating a first thin volume associated with a production volume;
computer program code for generating a first replica of the production volume by copying data from the production volume to a replica volume, wherein the copying is performed as a background process while processing new received I/O requests;
during the copying:
computer program code for receiving an I/O request to be written to the production volume;
computer program code for writing data from the I/O request to the first thin volume and tracking data changed due to the I/O request in metadata associated with the production volume and the first thin volume;
computer program code for checking data size of the first thin volume, and, when the size of the first thin volume is below a threshold, asynchronously applying changes from the first thin volume to a backup storage;
when the size of the first thin volume is at or above a threshold:
determining whether a region of the production volume associated with the received I/O request has already been copied from the production volume to the first thin volume; and
when the region is already copied, overwriting an associated region in the first thin volume, whereby an amount of metadata to track changed data is reduced.
13. The computer program product of claim 12 , further comprising, computer program code for performing the copying as a background process while processing new received I/O requests.
14. The computer program product of claim 12 , further comprising, when the size of the first thin volume is below the threshold and the region is already copied from the production volume to the replica volume:
when the copying data from the production volume to the replica volume is complete, computer program code for generating a second thin volume and writing data for new received I/O requests to the second thin volume.
15. The computer program product of claim 14 , further comprising, when the size of the first thin volume is at or above the threshold and the region is already copied from the production volume to the replica volume:
computer program code for copying data from the first thin volume to the replica volume;
during the copying from the first thin volume to the replica volume, computer program code for tracking data changed due to a new I/O request in metadata associated with the first thin volume; and
when the copying from the first thin volume to the replica volume is complete, computer program code for setting, based upon the metadata, locations as dirty data to be copied from the production volume to the replica volume.
16. The computer program product of claim 12 , further comprising, when the region of the production volume associated with the received I/O request has not yet been copied from the production volume to the replica volume, computer program code for tracking data changed due to the I/O request in metadata associated with the first thin volume.