Automatic consistency group scale-out for any point in time replication systems
One example method includes measuring a journal distribution lag of a journal, evaluating a first operational measure at a production site, and evaluating a second operational measure at a replication site operable to communicate with a production site, based on the journal distribution lag, and the evaluation of the first operational measure and the second operational measure, determining a scale out for a consistency group of the replication site, and implementing the scale out of the consistency group.
1. A method, comprising: measuring a journal distribution lag of a journal, wherein the journal distribution lag is a measure to determine an amount of data that has reached a replication appliance but has not yet been distributed by the replication appliance to the journal;
evaluating a first operational measure at a production appliance, and evaluating a second operational measure at the replication appliance, which is operable to communicate with the production appliance;
determining a scale out for a consistency group of the replication appliance is needed based on the journal distribution lag exceeding a threshold, the first operational measure, and the second operational measure, wherein the threshold defines an amount of memory available to the replication appliance of the consistency group;
implementing the scale out of the consistency group;
wherein when the scale out is implemented at a time ‘T,’ a barrier point is defined and inserted in the journal at the time ‘T,’ and the replication appliance of the consistency group is required to stop writing to the journal when the barrier point is reached;
and wherein after all replication appliances in the consistency group have reached the barrier point in their respective journals, and partitions of an address space corresponding to the consistency group have been updated, the replication appliances are permitted to begin, or resume, writing to their respective journals.
2. The method as recited in claim 1 , wherein when any of the operational measures exceeds a defined upper threshold, implementing the scale out comprises increasing a number of replication appliances of the consistency group, and when all of the operational measures are below a defined lower threshold, implementing the scale out comprises reducing the number of replication appliances of the consistency group.
3. The method as recited in claim 1 , wherein the first operational measure and/or the second operational measure each comprise one or more of: latency; IOPs; and, throughput.
4. The method as recited in claim 1 , wherein when the journal distribution lag exceeds the threshold for a defined period of time, scale out is determined to be needed, and when the journal distribution lag is at or below the threshold, scaled out is determined not to be needed.
5. The method as recited in claim 1 , wherein scale out of the consistency groups comprises either increasing, or decreasing, a number of replication appliances in the consistency group.
6. The method as recited in claim 1 , wherein the journal is operable to write data to a corresponding partition of an address space that contains all addresses to which replication appliances of the consistency group write.
7. The method as recited in claim 1 , wherein when scale out is implemented, one or more partitions of an address space that corresponds to the consistency group are modified.
8. A computer readable storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
measuring a journal distribution lag of a journal, wherein the journal distribution lag is a measure to determine an amount of data that has reached a replication appliance but has not yet been distributed by the replication appliance to the journal;
evaluating a first operational measure at a production appliance, and evaluating a second operational measure at the replication appliance, which is operable to communicate with the production appliance;
determining a scale out for a consistency group of the replication appliance is needed based on the first operational measure, the second operational measure, and the journal distribution lag exceeding a threshold, wherein the threshold defines an amount of memory available to the replication appliance of the consistency group; and
implementing the scale out of the consistency group;
wherein when the scale out is implemented at a time ‘T,’ a barrier point is defined and inserted in the journal at the time ‘T,’ and the replication appliance of the consistency group is required to stop writing to the journal when the barrier point is reached;
and wherein after all replication appliances in the consistency group have reached the barrier point in their respective journals, and partitions of an address space corresponding to the consistency group have been updated, the replication appliances are permitted to begin, or resume, writing to their respective journals.
9. The computer readable storage medium as recited in claim 8 , wherein when any of the operational measures exceeds a defined upper threshold, implementing the scale out comprises increasing a number of replication appliances of the consistency group, and when all of the operational measures are below a defined lower threshold, implementing the scale out comprises reducing the number of replication appliances of the consistency group.
10. The computer readable storage medium as recited in claim 8 , wherein the first operational measure and/or the second operational measure each comprise one or more of: latency; IOPs; and, throughput.
11. The computer readable storage medium as recited in claim 8 , wherein when the journal distribution lag exceeds the threshold for a defined period of time, scale out is determined to be needed, and when the journal distribution lag is at or below the threshold, scaled out is determined not to be needed.
12. The computer readable storage medium as recited in claim 8 , wherein scale out of the consistency groups comprises either increasing, or decreasing, a number of replication appliances in the consistency group.
13. The computer readable storage medium as recited in claim 8 , wherein the journal is operable to write data to a corresponding partition of an address space that contains all addresses to which replication appliances of the consistency group write.
14. The computer readable storage medium as recited in claim 8 , wherein when scale out is implemented, one or more partitions of an address space that corresponds to the consistency group are modified.