Data replication in data storage systems
The present invention relates to system and method of asynchronous data replication. In an embodiment, the system includes a primary host that asynchronously transmits write and sync commands where each write and sync command includes a data set ID and a sequence number. A secondary host coupled to the primary host receives the write and sync commands out of order, tracks receipt of write and sync commands, and after receipt of the write and sync commands of the data set writes blocks associated with the write commands to data storage coupled to the secondary host.
1. A method of asynchronous data replication over a network, wherein one or more applications send write commands to a primary host, comprising:
at the primary host:
acknowledging each write command before transmitting the write command to a secondary host;
assigning a data set ID to each write command;
transmitting each write command before the application has sent all of the write commands of the data set to the primary host;
assigning the data set ID to a sync command;
transmitting the sync command, wherein the write commands and the sync command define the data set;
at the secondary host:
receiving the write commands and sync command out of order; and
writing the data set to a storage coupled to the secondary host.
2. The method of claim 1 , further comprising tracking the receipt of the write and sync commands at the secondary host.
3. The method of claim 2 , further comprising assigning a unique sequence number for each write command and another sequence number to the sync command at the primary host.
4. The method of claim 3 , wherein tracking the receipt of the write and sync commands includes setting bits in an identifier bitmap corresponding to the data set at the secondary host.
5. The method of claim 4 , further comprising using the identifier bitmap to determine receipt of the data set at the secondary host.
6. The method of claim 1 , further comprising writing a block of the data set to a secondary target VLUN and acknowledging the write and sync commands before receipt of the data set at the secondary host.
7. The method of claim 5 , further comprising updating a data structure at the secondary host to track blocks of the data set in the secondary target VLUN.
8. The method of claim 7 , wherein updating the data structure at the secondary host includes setting bits in a bitmap and updating a log file.
9. The method of claim 1 , further comprising storing multiple versions of one of the blocks of the data set at the primary host while waiting for an acknowledgement from the secondary host.
10. The method of claim 1 , further comprising writing blocks of the data set to a memory buffer at the secondary host and acknowledging the write and sync commands at the secondary host before receipt of the data set.
11. The method of claim 1 , wherein the step of receiving at the secondary host includes receiving write commands belonging to a plurality of data sets that are overlapping with respect to each other.
12. The method of claim 11 , further comprising storing the data sets in a plurality of data structures.
13. The method of claim 1 , further comprising receiving a second data set at the secondary host, wherein the step of receiving write commands for the second data set precedes receiving the write commands of the data set.
14. The method of claim 1 , further comprising transmitting the write and sync commands on the network using the Fibre channel protocol.
15. The method of claim 1 , further comprising transmitting the write and sync commands on the network using the iSCSI protocol.
16. The method of claim 1 , wherein the step of writing includes the secondary host writing the blocks of the data set to a secondary source VLUN.
17. The method of claim 1 , wherein the data set contains one or more file system transactions.