Migrating data objects together with their snaps
View Patent ↗A technique for performing non-disruptive migration coordinates object migration with snapshot-shipping to migrate both a data object and its snaps from a source to a target. Snapshot-shipping conveys snaps to the target, and an internal snap of the data object serves as a basis for building a migrated version of the data object at the target. As IO requests specifying writes to the data object arrive at the source, a data mirroring operation writes the arriving data both to the data object at the source and to the version thereof at the target. In parallel with the data mirroring operation, a filtering copy operation copies data of the internal snap to the target, but avoids overwriting data mirrored to the target after the internal snap is taken.
1. A method of performing data migration, the method comprising:
duplicating multiple source snaps to generate multiple respective target snaps, each of the source snaps being a point-in-time version of a source data object;
generating an internal snap of the source data object, the internal snap being a point-in-time version of the source data object;
performing a data mirroring operation, the data mirroring operation including (i) receiving, over time, multiple IO (Input/Output) requests specifying respective data to be written to the source data object, (ii) writing the data specified in the IO requests both to the source data object and to a target data object, the target data object providing a migrated version of the source data object, and (iii) marking a bitmap to indicate locations within the source data object at which the data specified in the IO requests are written; and
while performing the data mirroring operation, simultaneously performing a filtering copy operation, the filtering copy operation including (i) in response to the bitmap not being marked for a first location within the source data object, copying a set of data at a corresponding first location of the internal snap to the target data object, and (ii) in response to the bitmap being marked for a second location within the source data object, performing no copying of data at a corresponding second location of the internal snap to the target data object,
wherein the method further comprises:
prior to generating the internal snap, pausing a processing of IO requests directed to the source data object;
while the processing of IO requests is paused, initializing the bitmap to reflect no marked locations within the source data object; and
after generating the internal snap and initializing the bitmap, resuming the processing of IO requests directed to the source data object.
2. The method of claim 1 , wherein migration of the source data object and its snaps is performed under direction of a mobility manager and in response to a migration request, the mobility manager coordinating activities of a snapshot-shipping transport, for duplicating the multiple source snaps, and a migration driver, for performing the data mirroring operation and the filtering copy operation.
3. The method of claim 2 , wherein, when performing the filtering copy operation, copying the set of data to the target data object includes:
identifying, by a source copying agent, a predetermined pattern in the set of data;
sending a code from the source copying agent to a target copying agent, the code representing the predetermined pattern and distinguishing from other patterns;
generating, by the target copy agent, the predetermined pattern represented by the code; and
storing the generated predetermined pattern as the set of data in the target object.
4. The method of claim 2 , wherein the source data object and the target data object are both stored within a single data storage system, wherein the data storage system has a cache, and wherein, when performing the filtering copy operation, copying the set of data to the target data object includes:
reading the set of data into the cache; and
writing the set of data from cache to the target data object.
5. The method of claim 4 , wherein the cache is mirrored between a first storage processor and a second storage processor of the data storage system, wherein reading the set of data into the cache is performed by the first storage processor, and wherein writing the set of data from cache into the target data object is performed by the second storage processor.
6. The method of claim 1 , wherein duplicating the multiple source snaps is performed with snapshot-shipping operations, the snapshot shipping operations including:
copying a first source snap to generate a target object, the first snap being an earliest point-in-time version of the source data object from among the multiple source snaps;
generating a first target snap of the target object; and
for each additional source snap going in order from earliest to most recent, computing a difference relative to an immediately previous source snap, applying the difference to the target object, and snapping the target object to create a respective target snap.
7. The method of claim 6 , wherein duplicating the multiple source snaps is performed after generating the internal snap.
8. The method of claim 6 , wherein duplicating the multiple source snaps is performed prior to generating the internal snap.
9. The method of claim 8 , further comprising computing a last difference between the internal snap and a most recent source snap, the last difference identifying data of the internal snap that differ from data of the most recent source snap and locations where those differences occur, wherein, when performing the filtering copy operation, only locations indicated by the last difference are candidates for copying to the target data object based on whether the bitmap is marked.
10. A data storage system, comprising control circuitry that includes a set of processing units coupled to memory, the control circuitry constructed and arranged to:
duplicate multiple source snaps to generate multiple respective target snaps, each of the source snaps being a point-in-time version of a source data object;
generate an internal snap of the source data object, the internal snap being a point-in-time version of the source data object;
perform a data mirroring operation, the data mirroring operation including (i) receiving, over time, multiple IO (Input/Output) requests specifying respective data to be written to the source data object, (ii) writing the data specified in the IO requests both to the source data object and to a target data object, the target data object providing a migrated version of the source data object, and (iii) marking a bitmap to indicate locations within the source data object at which the data specified in the IO requests are written; and
while performing the data mirroring operation, simultaneously perform a filtering copy operation, the filtering copy operation including (i) in response to the bitmap not being marked for a first location within the source data object, copying a set of data at a corresponding first location of the internal snap to the target data object, and (ii) in response to the bitmap being marked for a second location within the source data object, performing no copying of data at a corresponding second location of the internal snap to the target data object,
wherein the control circuitry constructed and arranged to duplicate the multiple source snaps is further constructed and arranged to:
copy a first source snap to generate a target object, the first snap being an earnest point-in-time version of the source data object from among the multiple source snaps;
generate a first target snap of the target object; and
for each additional source snap going in order from earliest to most recent, compute a difference relative to an immediately previous source snap, apply the difference to the target object, and snap the target object to create a respective target snap.
11. A computer program product including a set of non-transitory, computer-readable media having instructions which, when executed by control circuitry of a data storage system, cause the control circuitry to perform a data migration method, the method comprising:
duplicating multiple source snaps to generate multiple respective target snaps, each of the source snaps being a point-in-time version of a source data object;
generating an internal snap of the source data object, the internal snap being a point-in-time version of the source data object;
performing a data mirroring operation, the data mirroring operation including (i) receiving, over time, multiple IO (Input/Output) requests specifying respective data to be written to the source data object, (ii) writing the data specified in the IO requests both to the source data object and to a target data object, the target data object providing a migrated version of the source data object, and (iii) marking a bitmap to indicate locations within the source data object at which the data specified in the IO requests are written; and
while performing the data mirroring operation, simultaneously performing a filtering copy operation, the filtering copy operation including (i) in response to the bitmap not being marked for a first location within the source data object, copying a set of data at a corresponding first location of the internal snap to the target data object, and (ii) in response to the bitmap being marked for a second location within the source data object, performing no copying of data at a corresponding second location of the internal snap to the target data object,
wherein, when performing the filtering copy operation, copying the set of data to the target data object includes:
identifying, by a source copying agent, a predetermined pattern in the set of data;
sending a code from the source copying agent to a target copying agent, the code representing the predetermined pattern and distinguishing from other patterns;
generating, by the target copy agent, the predetermined pattern represented by the code; and
storing the generated predetermined pattern as the set of data in the target object.
12. The computer program product of claim 11 , wherein migration of the source data object and its snaps is performed under direction of a mobility manager and in response to a migration request, the mobility manager coordinating activities of a snapshot-shipping transport, for duplicating the multiple source snaps, and a migration driver, for performing the data mirroring operation and the filtering copy operation.
13. The computer program product of claim 12 , wherein duplicating the multiple source snaps by the snapshot-shipping transport includes:
copying a first source snap to generate a target object, the first snap being an earliest point-in-time version of the source data object from among the multiple source snaps;
generating a first target snap of the target object; and
for each additional source snap going in order from earliest to most recent, computing a difference relative to an immediately previous source snap, applying the difference to the target object, and snapping the target object to create a respective target snap.
14. The computer program product of claim 13 , wherein the migration request received by the mobility manager specifies an option for invoking the snapshot-shipping transport before invoking the migration driver, and wherein the mobility manager, in response to the option in the migration request, invokes the snap-shot shipping transport for duplicating the multiple source snaps prior to directing generation of the internal snap.
15. The computer program product of claim 14 , further comprising computing a last difference between the internal snap and a most recent source snap, the last difference identifying data of the internal snap that differ from data of the most recent source snap and locations where those differences occur, wherein, when performing the filtering copy operation, only locations indicated by the last difference are candidates for copying to the target data object based on whether the bitmap is marked.
16. The computer program product of claim 13 , wherein the migration request received by the mobility manager specifies an option for invoking the snapshot-shipping transport after invoking the migration driver, and wherein the mobility manager, in response to the option in the migration request, directs duplicating the multiple source snaps after directing generation of the internal snap.
17. The computer program product of claim 12 , wherein the source data object and the target data object are both stored within a single data storage system, wherein the data storage system has a cache, and wherein, when performing the filtering copy operation, copying the set of data to the target data object includes:
reading the set of data into the cache; and
writing the set of data from cache to the target data object.
18. The computer program product of claim 17 , wherein the cache is mirrored between a first storage processor and a second storage processor of the data storage system, wherein reading the set of data into the cache is performed by the first storage processor, and wherein writing the set of data from cache into the target data object is performed by the second storage processor.