Taking a consistent cut during replication for storage across multiple nodes without blocking input/output
A replication appliance maintains dependent-write consistency on replicated storage, for write operations made by multiple nodes to a primary storage destination, without blocking storage I/O operations. The replication appliance transmits a notification to each node, indicating to check a centralized location for a token prior to each I/O operation. The replication appliance atomically generates the token, and the nodes detect it prior to executing corresponding I/O operations. In response to detecting the token, each detecting node inserts a consistency marker in its I/O stream, proceeds with the corresponding I/O operation, and stops checking for the token. The appliance reads the I/O stream of each node, and identifies the position of the inserted consistency marker. Using the identified positions in the I/O streams, the appliance creates a consistent cut without blocking I/O operations made by the nodes. The appliance uses the cut to maintain dependent-write consistency to the replicated storage.
1. A computer implemented method for maintaining dependent-write consistency to replicated storage corresponding to a primary storage destination, for write operations made by multiple ones of a group of storage Input/Output (I/O) sources to the primary storage destination, without blocking storage I/O operations to the primary storage destination, the method comprising the steps of:
transmitting, by a centralized computing device, a notification to each storage I/O source in the group, the notification indicating to check a centralized location for presence of a token prior to each storage I/O operation to the primary storage destination;
atomically generating the token by the centralized computing device at the centralized location, such that checking the centralized location prior to a storage I/O operation to the primary storage destination by the multiple ones of the group of storage I/O sources results in I/O sources detecting the presence of the token prior to executing corresponding storage I/O operations to the primary storage destination, the presence of the token indicating to each given detecting storage I/O source to insert a consistency marker in its storage I/O stream, proceed with the corresponding storage I/O operation to the primary storage destination, and stop checking for the presence of the token prior to subsequent storage I/O operations to the primary storage destination;
reading the storage I/O stream of each storage I/O source of the group, by the centralized computing device, and identifying a position of an inserted consistency marker in each read storage I/O stream; and
creating, by the centralized computing device, a consistent cut that captures an ordered state of storage I/O operations to the primary storage destination at a specific point in time made by the multiple ones of the group of storage I/O sources, without blocking storage I/O operations made by the storage I/O sources of the group to the primary storage destination, using the positions of the inserted consistency markers in the storage I/O streams of the multiple ones of the group of storage I/O sources.
2. The method of claim 1 further comprising:
receiving asynchronously, by the centralized computing device, storage I/O operations executed by the multiple ones of the group of storage I/O sources to the primary storage destination, the received storage I/O operations to be executed by the centralized computing device to the replicated storage, wherein storage I/O operations to the primary storage destination received from multiples ones of the group of storage I/O sources represent different points in time; and
using the consistent cut to maintain dependent-write consistency between the multiple ones of the group to the replicated storage, by the centralized computing device.
3. The method of claim 1 wherein:
the group of storage I/O sources further comprises a plurality of nodes in a clustering and storage environment; and
the primary storage destination further comprises shared storage utilized within the clustering and storage environment.
4. The method of claim 1 wherein the replicated storage further comprises:
at least one volume in a clustering and storage environment being replicated to a secondary site.
5. The method of claim 1 wherein the primary storage destination further comprises:
a replicated file system.
6. The method of claim 1 wherein the primary storage destination further comprises:
at least one virtual storage device.
7. The method of claim 1 wherein the primary storage destination further comprises:
at least one physical storage device.
8. The method of claim 1 wherein:
the group of storage I/O sources further comprises a plurality of virtual machines running at a physical computer; and
the primary storage destination further comprises a virtual machine disk files accessed by the virtual machines.
9. The method of claim 1 wherein the centralized computing device further comprises:
a replication appliance.
10. The method of claim 1 further comprising:
transmitting, by the centralized computing device, the notification to each storage I/O source in the group, the notification indicating to any storage I/O source that does not have any storage I/O operations to the primary storage destination to execute to periodically poll the centralized location for the presence of the token; and
atomically generating the token by the centralized computing device, such that polling the centralized location by a storage I/O source results in the storage I/O source detecting the presence of the token, the presence of the token indicating to the storage I/O source to insert a consistency marker in its storage I/O stream and stop polling for the presence of the token.
11. The method of claim 1 further comprising:
subsequently to the multiple ones of the group of storage I/O sources having detected the presence of the token at the centralized location, deleting the token prior to creating a second consistent cut.
12. The method of claim 1 further comprising:
subsequently creating a second consistent cut, by the centralized computing device, by repeating the steps of transmitting the notification, atomically generating the token and reading the storage I/O stream of each storage I/O source of the group.
13. The method of claim 1 further comprising:
receiving, by each storage I/O sources of the group, a notification to go into token searching mode;
checking, by the multiple ones of the group of storage I/O sources, the centralized location for the presence of the atomically generated token before executing each storage I/O operation to the primary storage destination; and
in response to detecting the presence of the atomically generated token at the centralized location by a specific storage I/O source prior to executing a specific storage I/O operation to the primary storage destination, the specific storage I/O source inserting a consistency marker in a storage I/O stream of the specific storage I/O source; and
subsequently to the specific storage I/O source inserting the consistency marker in its storage I/O stream, executing the specific storage I/O operation to the primary storage destination, by the specific storage I/O source.
14. At least one non-transitory computer readable medium for maintaining dependent-write consistency to replicated storage corresponding to a primary storage destination, for write operations made by multiple ones of a group of storage Input/Output (I/O) sources to the primary storage destination, without blocking storage I/O operations to the primary storage destination, the at least one non-transitory computer readable medium storing computer executable instructions that, when loaded into computer memory and executed by at least one processor of a computing device, cause the computing device to perform the following steps:
transmitting, by a centralized computing device, a notification to each storage I/O source in the group, the notification indicating to check a centralized location for presence of a token prior to each storage I/O operation to the primary storage destination;
atomically generating the token by the centralized computing device at the centralized location, such that checking the centralized location prior to a storage I/O operation to the primary storage destination by multiple ones of the group of storage I/O sources results in I/O sources detecting the presence of the token prior to executing corresponding storage I/O operations to the primary storage destination, the presence of the token indicating to each given detecting storage I/O source to insert a consistency marker in its storage I/O stream, proceed with the corresponding storage I/O operation to the primary storage destination, and stop checking for the presence of the token prior to subsequent storage I/O operations to the primary storage destination;
reading the storage I/O stream of each storage I/O source of the group, by the centralized computing device, and identifying a position of an inserted consistency marker in each read storage I/O stream; and
creating, by the centralized computing device, a consistent cut that captures an ordered state of storage I/O operations to the primary storage destination at a specific point in time made by multiple ones of the group of storage I/O sources, without blocking storage I/O operations made by the storage I/O sources of the group to the primary storage destination, using the identified positions of the inserted consistency markers in the storage I/O streams of multiple ones of the group of storage I/O sources.
15. The at least one non-transitory computer readable medium of claim 14 further comprising:
receiving asynchronously, by the centralized computing device, storage I/O operations executed by the multiple ones of the group of storage I/O sources to the primary storage destination, the received storage I/O operations to be executed by the centralized computing device to the replicated storage, wherein storage I/O operations to the primary storage destination received from multiples ones of the group of storage I/O sources represent different points in time; and
using the consistent cut to maintain dependent-write consistency between the multiple ones of the group to the replicated storage, by the centralized computing device.
16. The at least one non-transitory computer readable medium of claim 14 wherein:
the group of storage I/O sources further comprises a plurality of nodes in a clustering and storage environment; and
the primary storage destination further comprises shared storage utilized within the clustering and storage environment.
17. The at least one non-transitory computer readable medium of claim 14 further comprising:
transmitting, by the centralized computing device, the notification to each storage I/O source in the group, the notification indicating to any storage I/O source that does not have any storage I/O operations to the primary storage destination to execute to periodically poll the centralized location for the presence of the token; and
atomically generating the token by the centralized computing device, such that polling the centralized location by a storage I/O source results in the storage I/O source detecting the presence of the token, the presence of the token indicating to the storage I/O source to insert a consistency marker in its storage I/O stream and stop polling for the presence of the token.
18. The at least one non-transitory computer readable medium of claim 14 further comprising:
subsequently to multiple ones of the group of storage I/O sources having detected the presence of the token at the centralized location, deleting the token prior to creating a second consistent cut.
19. The at least one non-transitory computer readable medium of claim 14 further comprising:
subsequently creating a second consistent cut, by the centralized computing device, by repeating the steps of transmitting the notification, atomically generating the token and reading the storage I/O stream of each storage I/O source of the group.
20. A computer system for maintaining dependent-write consistency to replicated storage corresponding to a primary storage destination, for write operations made by multiple ones of a group of storage Input/Output (I/O) sources to the primary storage destination, without blocking storage I/O operations to the primary storage destination, the computer system comprising:
a processor;
system memory;
a notification transmitting module residing in the system memory, the notification transmitting module being programmed to transmit a notification from a centralized computing device to each storage I/O source in the group, the notification indicating to check a centralized location for presence of a token prior to each storage I/O operation to the primary storage destination;
a token generating module residing in the system memory, the token generating module being programmed to atomically generate the token at the centralized location, such that checking the centralized location prior to a storage I/O operation to the primary storage destination by multiple ones of the group of storage I/O sources results in I/O sources detecting the presence of the token prior to executing corresponding storage I/O operations to the primary storage destination, the presence of the token indicating to each given detecting storage I/O source to insert a consistency marker in its storage I/O stream, proceed with the corresponding storage I/O operation to the primary storage destination, and stop checking for the presence of the token prior to subsequent storage I/O operations to the primary storage destination;
a storage I/O stream reading module residing in the system memory, the storage I/O stream reading module being programmed to read the storage I/O stream of each storage I/O source of the group, and to identify a position of an inserted consistency marker in each read storage I/O stream; and
a consistent cut creating module residing in the system memory, the consistent cut creating module being programmed to create a consistent cut that captures an ordered state of storage I/O operations to the primary storage destination at a specific point in time made by multiple ones of the group of storage I/O sources, without blocking storage I/O operations made by the storage I/O sources of the group to the primary storage destination, using the positions of the inserted consistency markers in the storage I/O streams of multiple ones of the group of storage I/O sources.