Synchronization of snapshots in a distributed consistency group
Techniques for synchronizing snapshot operations in a distributed consistency group of storage volumes. The disclosed techniques employ a schedule of discrete snapshot operations (e.g., pausing I/O requests from host computers, taking snapshots of storage volumes, un-pausing the I/O requests), which is provided to storage appliances storing the storage volumes within the distributed consistency group. By pre-scheduling such discrete snapshot operations to be performed by the storage appliances, an amount of real-time communications (e.g., issuing snapshot commands, receiving snapshot acknowledgements) required to complete the snapshot operations is reduced, allowing data storage systems to more easily satisfy I/O time-out requirements and avoid application unresponsiveness and/or crashes.
1. A method of synchronizing snapshot operations in a distributed consistency group of storage volumes, comprising:
generating a schedule for performing two or more discrete snapshot operations at a plurality of storage appliances that store a plurality of storage volumes, respectively, in the distributed consistency group, the schedule indicating predetermined times when the respective discrete snapshot operations are to be performed at the plurality of storage appliances;
sending the schedule to each of the plurality of storage appliances;
performing the two or more discrete snapshot operations at each of the plurality of storage appliances at the predetermined times indicated by the schedule to create a plurality of snapshots of the plurality of storage volumes, respectively, in the distributed consistency group; and
reconciling the plurality of snapshots of the respective storage volumes in the distributed consistency group to assure their validity,
wherein each snapshot among the plurality of snapshots has associated metadata, the associated metadata including a first timestamp indicating a first time when input/output (I/O) requests were paused at a respective storage appliance where the snapshot was taken, and a second timestamp indicating a second time when the snapshot was taken at the respective storage appliance, and
wherein the reconciling of the plurality of snapshots of the respective storage volumes in the distributed consistency group includes determining whether the first timestamp and the second timestamp align to provide a point-in-time (PIT) replication of the respective storage volumes.
2. The method of claim 1 wherein the generating of the schedule includes generating the schedule for performing at least a first operation at a first predetermined time for pausing I/O requests at the respective storage appliances, and a second operation at a second predetermined time for taking snapshots of storage volumes stored by the respective storage appliances.
3. The method of claim 2 wherein the generating of the schedule includes incorporating, in the schedule, a delay prior to the second predetermined time to assure that the first operation is completed at the respective storage appliances before the second operation is performed at the respective storage appliances.
4. The method of claim 2 wherein the generating of the schedule includes generating the schedule for performing a third operation at a third predetermined time for un-pausing the I/O requests at the respective storage appliances.
5. The method of claim 1 further comprising:
time synchronizing the plurality of storage appliances by a network time protocol (NTP) server.
6. The method of claim 1 further comprising:
receiving, from the plurality of storage appliances, a plurality of snapshot acknowledgements including the plurality of snapshots, respectively, in the distributed consistency group.
7. A system for synchronizing snapshot operations in a distributed consistency group of storage volumes, comprising:
a computerized snapshot management system;
a plurality of storage appliances configured to store a plurality of storage volumes, respectively, in the distributed consistency group; and
a network configured to interconnect the computerized snapshot management system and the plurality of storage appliances,
wherein the computerized snapshot management system is operative:
to generate a schedule for performing two or more discrete snapshot operations at the plurality of storage appliances, the schedule indicating predetermined times when the respective discrete snapshot operations are to be performed at the plurality of storage appliances; and
to send the schedule to each of the plurality of storage appliances over the network,
wherein each storage appliance among the plurality of storage appliances is operative:
to perform the two or more discrete snapshot operations at the predetermined times indicated by the schedule to create a snapshot of a respective storage volume in the distributed consistency group stored by the storage appliance,
wherein the snapshot of the respective storage volume has associated metadata, the associated metadata including a first timestamp indicating a first time when input/output (I/O) requests were paused at the storage appliance where the snapshot was taken, and a second timestamp indicating a second time when the snapshot was taken at the storage appliance, and
wherein the computerized snapshot management system is further operative:
to reconcile the snapshot of the respective storage volume in the distributed consistency group to assure its validity, including determining whether the first timestamp and the second timestamp align to provide a point-in-time (PIT) replication of the respective storage volume.
8. The system of claim 7 wherein the computerized snapshot management system is further operative to generate the schedule for performing at least a first operation at a first predetermined time for pausing the I/O requests at the storage appliance, and a second operation at a second predetermined time for taking the snapshot of the respective storage volume stored by the storage appliance.
9. The system of claim 8 wherein the computerized snapshot management system is further operative to incorporate, in the schedule, a delay prior to the second predetermined time to assure that the first operation is completed at the storage appliance before the second operation is performed at the storage appliance.
10. The system of claim 8 wherein the computerized snapshot management system is further operative to generate the schedule for performing a third operation at a third predetermined time for un-pausing the I/O requests at the storage appliance.
11. The system of claim 7 further comprising:
a network time protocol (NTP) server connected to the network, the NTP server being operative to time synchronize the plurality of storage appliances.
12. The system of claim 7 wherein the computerized snapshot management system is further operative to receive, from the plurality of storage appliances, a plurality of snapshot acknowledgements including the plurality of snapshots, respectively, in the distributed consistency group.
13. The system of claim 7 wherein the network is one or more of a local area network (LAN), an Ethernet sub-network, a PCI or PCIe sub-network, a switched PCIe sub-network, a wide area network (WAN), a metropolitan area network (MAN), and the Internet.
14. The system of claim 7 wherein the plurality of storage volumes are one of logical storage units (LUNs), sub-LUNs, and virtual machine volumes (VVOLs).
15. A computer program product having a non-transitory computer readable medium that stores a set of instructions to synchronize snapshot operations in a distributed consistency group of storage volumes, the set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of:
generating a schedule for performing two or more discrete snapshot operations at a plurality of storage appliances that store a plurality of storage volumes, respectively, in the distributed consistency group, the schedule indicating predetermined times when the respective discrete snapshot operations are to be performed at the plurality of storage appliances;
sending the schedule to each of the plurality of storage appliances;
performing the two or more discrete snapshot operations at each of the plurality of storage appliances at the predetermined times indicated by the schedule to create a plurality of snapshots of the plurality of storage volumes, respectively, in the distributed consistency group; and
reconciling the plurality of snapshots of the respective storage volumes in the distributed consistency group to assure their validity
wherein each snapshot among the plurality of snapshots has associated metadata, the associated metadata including a first timestamp indicating a first time when input/output (I/O) requests were paused at a respective storage appliance where the snapshot was taken, and a second timestamp indicating a second time when the snapshot was taken at the respective storage appliance, and
wherein the reconciling of the plurality of snapshots of the respective storage volumes in the distributed consistency group includes determining whether the first timestamp and the second timestamp align to provide a point-in-time (PIT) replication of the respective storage volumes.
16. The method of claim 15 wherein the generating of the schedule includes generating the schedule for performing at least a first operation at a first predetermined time for pausing I/O requests at the respective storage appliances, and a second operation at a second predetermined time for taking snapshots of storage volumes stored by the respective storage appliances.
17. The method of claim 16 wherein the generating of the schedule includes incorporating, in the schedule, a delay prior to the second predetermined time to assure that the first operation is completed at the respective storage appliances before the second operation is performed at the respective storage appliances.
18. The method of claim 16 wherein the generating of the schedule includes generating the schedule for performing a third operation at a third predetermined time for un-pausing the I/O requests at the respective storage appliances.