Methods and systems for data resynchronization in a replication environment
Methods, computer program products, computer systems, and the like are disclosed that provide for scalable deduplication in an efficient and effective manner. For example, such methods, computer program products, and computer systems can include determining whether a source data store and a replicated data store are unsynchronized and, in response to a determination that the source data store and the replicated data store are unsynchronized, performing a resynchronization operation. The source data stored in the source data store is replicated to replicated data in the replicated data store. The resynchronization operation resynchronizes the source data and the replicated data.
1. A method comprising:
detecting occurrence of an event, wherein
the event is one of a restart event or a startup event of a computing system at a target site; and
in response to detection of the occurrence of the event,
determining whether a source data store at a source site and a replicated data store at the target site are unsynchronized, wherein
source data stored in the source data store has been replicated to replicated data in the replicated data store,
the determining comprises
determining whether a data structure indicates that one or more units of data of the source data have not been successfully replicated to the replicated data store,
the data structure is associated with the source data store, and
the data structure comprises a merged synchronization data structure, and
in response to a determination that the source data store and the replicated data store are unsynchronized, performing a resynchronization operation,
wherein
the resynchronization operation resynchronizes the source data and the replicated data based, at least in part, on data in the data structure.
2. The method of claim 1 , wherein
the source data and the replicated data have become unsynchronized as a result of a restart condition caused by the restart event.
3. The method of claim 1 , wherein
the startup event is one of a failover operation, a restoration operation, or a reboot operation.
4. The method of claim 1 , wherein
the data structure is one of
an active data structure, or
a synchronization data structure.
5. The method of claim 3 , wherein the determining comprises:
comparing a first restore identifier and a second restore identifier, wherein
the first restore identifier is a last stored restore identifier associated with a state of the source data, and
the second restore identifier is a last replicated restore identifier associated with the state of the replicated data;
identifying an identified restore identifier based on a result of the comparing; and
sending the identified restore identifier from a source data mover to a target data mover.
6. The method of claim 5 , further comprising:
in response to the comparing indicating that the first restore identifier and the second restore identifier agree, performing the resynchronization operation from a first point in time represented by the first restore identifier; and
in response to the comparing indicating that the first restore identifier and the second restore identifier do not agree, performing the resynchronization operation from a second point in time represented by the second restore identifier.
7. The method of claim 5 , wherein the comparing is performed as part of communications between a write tracker and the source data mover, and the method further comprises:
retrieving retrieved metadata according to the identified restore identifier, wherein the retrieving is performed by the target data mover;
sending the retrieved metadata from the target data mover to the source data mover; and
providing the retrieved metadata to the write tracker.
8. The method of claim 7 , further comprising:
generating a merged synchronization data structure by merging the retrieved metadata into a synchronization data structure, wherein
the synchronization data structure is maintained by the write tracker, and the merging is performed by the write tracker;
sending the merged synchronization data structure to the target data mover; and for each element of the merged synchronization data structure,
retrieving a unit of the replicated data corresponding to the each element of the merged synchronization data structure, and
sending the unit of the replicated data from the target data mover to the source data mover.
9. The method of claim 7 , further comprising:
determining whether verification information for an element of the retrieved metadata indicates that a unit of data corresponding to the element of the retrieved metadata is unchanged; and
in response to a determination that the verification information indicates that the unit of data corresponding to the element of the retrieved metadata is unchanged, preventing sending of the unit of data corresponding to the element of the retrieved metadata from the target data mover to the write tracker.
10. The method of claim 1 , wherein
the source data and the replicated data are unsynchronized as a result of a startup condition.
11. The method of claim 10 , further comprising:
performing a synchronization operation;
upon completion of the synchronization operation, performing a replication operation,
wherein
the replication operation replicates a unit of data from the source data store to the replicated data store, and
the unit of data is subject to a write operation; and
maintaining an active data structure, wherein
the performing the replication operation and the maintaining the active data structure are performed concurrently.
12. The method of claim 11 , wherein the synchronization operation comprises:
selecting one or more elements of a synchronization data structure, wherein
the synchronization data structure comprises a plurality of elements, and each of the plurality of elements of the synchronization data structure indicate a write operation was performed on a corresponding unit of data of a plurality of units of data stored in the source data store;
reading one or more units of data of the plurality of units of data corresponding to the one or more elements of the synchronization data structure;
generating metadata corresponding to the one or more units of data; and
sending the one or more units of data and the metadata to a data mover.
13. A non-transitory computer-readable storage medium, comprising program instructions, which, when executed by one or more processors of a computing system, perform a method comprising:
detecting occurrence of an event, wherein
the event is one of a restart event or a startup event of another computing system, and
the another computing system is at a target site; and
in response to detection of the occurrence of the event,
determining whether a source data store at a source site and a replicated data store at the target site are unsynchronized, wherein
source data stored in the source data store has been replicated to replicated data in the replicated data store,
the determining comprises
determining whether a data structure indicates that one or more units of data of the source data have not been successfully replicated to the replicated data store,
the data structure is associated with the source data store, and
the data structure comprises a merged synchronization data structure, and
in response to a determination that the source data store and the replicated data store are unsynchronized, performing a resynchronization operation,
wherein
the resynchronization operation resynchronizes the source data and the replicated data based, at least in part, on data in the data structure.
14. The non-transitory computer-readable storage medium of claim 13 , wherein
the data structure is one of
an active data structure, or
a synchronization data structure.
15. The non-transitory computer-readable storage medium of claim 13 , wherein the source data and the replicated data have become unsynchronized as a result of restart condition and the determining comprises:
comparing a first restore identifier and a second restore identifier, wherein
the first restore identifier is a last stored restore identifier associated with a state of the source data, and
the second restore identifier is a last replicated restore identifier associated with the state of the replicated data;
identifying an identified restore identifier based on a result of the comparing; and
sending the identified restore identifier from a source data mover to a target data mover.
16. The non-transitory computer-readable storage medium of claim 15 , further comprising:
in response to the comparing indicating that the first restore identifier and the second restore identifier agree, performing the resynchronization operation from a first point in time represented by the first restore identifier; and
in response to the comparing indicating that the first restore identifier and the second restore identifier do not agree, performing the resynchronization operation from a second point in time represented by the second restore identifier.
17. The non-transitory computer-readable storage medium of claim 15 , wherein the comparing is performed as part of communications between a write tracker and the source data mover, and the method further comprises:
retrieving retrieved metadata according to the identified restore identifier, wherein the retrieving is performed by the target data mover;
sending the retrieved metadata from the target data mover to the source data mover; and
providing the retrieved metadata to the write tracker.
18. The non-transitory computer-readable storage medium of claim 17 , further comprising:
generating a merged synchronization data structure by merging the retrieved metadata into a synchronization data structure, wherein
the synchronization data structure is maintained by the write tracker, and the merging is performed by the write tracker;
sending the merged synchronization data structure to the target data mover; and for each element of the merged synchronization data structure,
retrieving a unit of the replicated data corresponding to the each element of the merged synchronization data structure, and
sending the unit of the replicated data from the target data mover to the source data mover.
19. The method of claim 17 , further comprising:
determining whether verification information for an element of the retrieved metadata indicates that a unit of data corresponding to the element of the retrieved metadata is unchanged; and
in response to a determination that the verification information indicates that the unit of data corresponding to the element of the retrieved metadata is unchanged, preventing sending of the unit of data corresponding to the element of the retrieved metadata from the target data mover to the write tracker.
20. A computing system comprising:
one or more processors; and
a computer-readable storage medium coupled to the one or more processors, comprising program instructions, which, when executed by the one or more processors, perform a method comprising
detecting occurrence of an event, wherein
the event is one of a restart event or a startup event of a computing system at a target site, and
in response to detection of the occurrence of the event,
determining whether a source data store at a source site and a replicated data store at the target site are unsynchronized, wherein source data stored in the source data store has been replicated to replicated data in the replicated data store,
the determining comprises
determining whether a data structure indicates that one or more units of data of the source data have not been successfully replicated to the replicated data store,
the data structure is associated with the source data store, and
the data structure comprises a merged synchronization data structure, and
in response to a determination that the source data store and the replicated data store are unsynchronized, performing a resynchronization operation, wherein
the resynchronization operation resynchronizes the source data and the replicated databased, at least in part, on data in the data structure.