Synchronous replication in a storage system
In one aspect, data replication processes include setting a minimum snap set creation interval, a maximum snap set creation interval, and a recovery time threshold (RTT). An aspect further includes creating snap sets during a synchronous replication process based on the snap set creation intervals and the RTT. The snap set creation further includes monitoring an amount of data changes since the last snap set creation, monitoring throughput statistics between a source and target system, and estimating an amount of time to replicate the data changes to the target system based on the data changes and throughput statistics. Upon determining the time to replicate the data changes reaches the RTT and the time since the last snap set creation passes the minimum interval, the snap set creation includes creating a next snap set, storing the next snap set at the source, and replicating the next snap set at the target.
1. A method, comprising:
setting, via a storage system, a minimum snap set creation interval, the minimum snap set creation interval specifying a minimum amount of elapsed time between a previously created snap set and a next snap set to be created;
setting, via the storage system, a maximum snap set creation interval, the maximum snap set creation interval specifying a maximum amount of elapsed time between the previously created snap set and the next snap set to be created, wherein the maximum amount of elapsed time is greater than the minimum amount of elapsed time;
setting, via the storage system, a recovery time threshold;
creating, at a source system of the storage system, snap sets during a synchronous replication process based on the minimum and maximum snap set creation intervals and the recovery time threshold, the creating snap sets comprising:
monitoring I/O operations to determine an amount of data changes since a last snap set creation, the amount of data changes being a measure of a difference in data via updates received by the source system;
monitoring throughput statistics between the source system and a target system of the storage system;
estimating an amount of time to replicate the data changes to the target system based on the amount of data changes since the last snap set creation and the throughput statistics;
upon determining both the time to replicate the data changes since the last snap set creation reaches the recovery time threshold and the time since the last snap set creation exceeds the minimum snap set creation interval:
creating a next snap set;
storing the next snap set at the source system; and
replicating the next snap set at the target system; and
upon determining the time to replicate the data changes does not reach the recovery time threshold, waiting until the maximum snap set creation interval has been reached before creating the next snap set.
2. The method of claim 1 , wherein the throughput statistics comprise a latency metric between the source system and the target system.
3. The method of claim 1 , wherein:
the recovery time threshold comprises a target time frame to reach a synchronous state during a recovery operation.
4. The method of claim 1 , wherein the amount of data changes comprises a measure of a difference in data via updates received by the source system, the measure of the difference tracked via a counter at the source system.
5. The method of claim 1 , further comprising, in response to a failover event:
performing, via the storage system, incremental synchronous replication between the source system and the target system.
6. The method of claim 1 , wherein the minimum and maximum snap set creation intervals and the recovery time threshold are user-tunable parameters.
7. A system, comprising:
a memory comprising computer-executable instructions; and
a processor operable by a storage system, the processor executing the computer-executable instructions, the computer-executable instructions when executed by the processor cause the processor to perform operations comprising:
setting a minimum snap set creation interval, the minimum snap set creation interval specifying a minimum amount of elapsed time between a previously created snap set and a next snap set to be created;
setting a maximum snap set creation interval, the maximum snap set creation interval specifying a maximum amount of elapsed time between the previously created snap set and the next snap set to be created, wherein the maximum amount of elapsed time is greater than the minimum amount of elapsed time;
setting a recovery time threshold;
creating, at a source system of the storage system, snap sets during a synchronous replication process based on the minimum and maximum snap set creation intervals and the recovery time threshold, the creating snap sets comprising:
monitoring I/O operations to determine an amount of data changes since a last snap set creation, the amount of data changes being a measure of a difference in data via updates received by the source system;
monitoring throughput statistics between the source system and a target system of the storage system;
estimating an amount of time to replicate the data changes to the target system based on the amount of data changes since the last snap set creation and the throughput statistics; and
upon determining both the time to replicate the data changes since the last snap set creation reaches the recovery time threshold and the time since the last snap set creation exceeds the minimum snap set creation interval:
creating a next snap set;
storing the next snap set at the source system; and
replicating the next snap set at the target system; and
upon determining the time to replicate the data changes does not reach the recovery time threshold, waiting until the maximum snap set creation interval has been reached before creating the next snap set.
8. The system of claim 7 , wherein the throughput statistics comprise a latency metric between the source system and the target system.
9. The system of claim 7 , wherein:
the recovery time threshold comprises a target time frame to reach a synchronous state during a recovery operation.
10. The system of claim 7 , wherein the amount of data changes comprises a measure of a difference in data via updates received by the source system, the measure of the difference tracked via a counter at the source system.
11. The system of claim 7 , further comprising, in response to a failover event, the operations further include:
performing incremental synchronous replication between the source system and the target system.
12. The system of claim 7 , wherein the minimum and maximum snap set creation intervals and the recovery time threshold are user-tunable parameters.
13. A computer program product embodied on a non-transitory computer readable medium, the computer program product including instructions that, when executed by a computer, causes the computer to perform operations comprising:
setting a minimum snap set creation interval, the minimum snap set creation interval specifying a minimum amount of elapsed time between a previously created snap set and a next snap set to be created;
setting a maximum snap set creation interval, the maximum snap set creation interval specifying a maximum amount of elapsed time between the previously created snap set and the next snap set to be created, wherein the maximum amount of elapsed time is greater than the minimum amount of elapsed time;
setting a recovery time threshold;
creating, at a source system of the storage system, snap sets during a synchronous replication process based on the minimum and maximum snap set creation intervals and the recovery time threshold, the creating snap sets comprising:
monitoring I/O operations to determine an amount of data changes since a last snap set creation, the amount of data changes being a measure of a difference in data via updates received by the source system;
monitoring throughput statistics between the source system and a target system of the storage system;
estimating an amount of time to replicate the data changes to the target system based on the amount of data changes since the last snap set creation and the throughput statistics;
upon determining both the time to replicate the data changes since the last snap set creation reaches the recovery time threshold and the time since the last snap set creation exceeds the minimum snap set creation interval:
creating a next snap set;
storing the next snap set at the source system; and
replicating the next snap set at the target system; and
upon determining the time to replicate the data changes does not reach the recovery time threshold, waiting until the maximum snap set creation interval has been reached before creating the next snap set.
14. The computer program product of claim 13 , wherein the throughput statistics comprise a latency metric between the source system and the target system.
15. The computer program product of claim 13 , wherein:
the recovery time threshold comprises a target time frame to reach a synchronous state during a recovery operation.
16. The computer program product of claim 13 , wherein the amount of data changes comprises a measure of a difference in data via updates received by the source system, the measure of the difference tracked via a counter at the source system.
17. The computer program product of claim 13 , further comprising, in response to a failover event, the operations further include:
performing incremental synchronous replication between the source system and the target system.