Data synchronization
A computer implemented method for reading signatures corresponding to a portion of data on a virtual machine disk on a production site, wherein reading the signature includes mapping the virtual machine disk offset into a physical storage offset, and reading the signature from the physical storage; sending the signature of the portion of data to a replication site; obtaining a mapping from a virtual disk to a physical disk at the replica site; issuing a command to write a portion of data corresponding to the signature to a the physical storage based on the mapping of the replica of the virtual machine disk on the replication site; determining if the command was successful; and based on a determination that the command was not successful, marking the locations corresponding to the signature for synchronization in a synchronization structure.
1. A computer implemented method comprising:
reading a signature corresponding to a portion of data on a virtual machine disk (VMDK) on a production site, wherein reading the signature includes obtaining a first mapping between a first block offset on the virtual machine disk on the production site and a second block offset on a first de-duplicated physical storage on the production site, and reading, based on the first mapping, the signature from the first de-duplicated physical storage on the production site:
sending the signature to a replication site;
obtaining a second mapping between a third block offset on a virtual machine disk on the replication site and a fourth block offset on a second de-duplicated physical disk storage on the replication site;
issuing a command to write the portion of data corresponding to the signature to the second de-duplicated physical storage on the replication site based on the second mapping;
determining, based at least in part on a result of the command, that a first set of blocks on the first de-duplicated physical storage on the production site and a second set of blocks on the second de-duplicated physical storage on the replication site, the first set of blocks and the second set of blocks corresponding to the signature, require synchronization; and
marking, in a synchronization structure, a first set of one or more locations on the first de-duplicated physical storage on the production site and a second set of one or more locations on the second de-duplicated physical storage on the replication site, the first set of one or more locations corresponding to the first set of blocks and the second set of one or more locations corresponding to the second set of blocks, the first set of blocks and the second set of blocks corresponding to the signature that require synchronization.
2. The method of claim 1 , wherein a splitter intercepts IOs for the virtual machine disk on the production site, and marks each location corresponding to the IOs arriving to the virtual machine disk on the production site as dirty in the synchronization structure.
3. The method of claim 1 , wherein a first snapshot of a first LU containing a first file system containing the virtual machine disk on the production site is created on the production site and a second snapshot of a second LU containing a second file system containing the virtual machine disk on the replication site is created on the replication site.
4. The method of claim 3 , wherein the first snapshot is attached to a first replication appliance at the production site and the second snapshot is attached to a second replication appliance at the replication site.
5. The method of claim 4 , wherein block hashes are read at the production site from the first LU using de-duplicated storage APIs.
6. The method of claim 5 , wherein the block hashes from the production site are used with an offset of the second VMDK by a write by Hash command.
7. The method of claim 6 , wherein the offset of the second LU used by the write by Hash command is based on the second mapping between the VMDK and the physical LU on the replication site.
8. The method of claim 7 , wherein if the write by Hash command fails, then each location associated with the failed write by Hash command is added to the synchronization structure for synchronization at a later point in time.
9. A computer program product comprising:
a non-transitory computer readable medium encoded with computer executable program code, wherein the code enables execution across one or more processors of:
reading a signature corresponding to a portion of data on a virtual machine disk (VMDK) on a production site, wherein reading the signature includes obtaining a first mapping between a first block offset on the virtual machine disk on the production site and a second block offset on a first de-duplicated physical storage on the production site, and reading, based on the first mapping, the signature from the first de-duplicated physical storage on the production site;
sending the signature to a replication site;
obtaining a second mapping between a third block offset on a virtual machine disk on the replication site and a fourth block offset on a second de-duplicated physical disk storage on the replication site;
issuing a command to write the portion of data corresponding to the signature to the second de-duplicated physical storage on the replication site based on the second mapping;
determining, based at least in part on a result of the command, that a first set of blocks on the first de-duplicated physical storage on the production site and a second set of blocks on the second de-duplicated physical storage on the replication site, the first set of blocks and the second set of blocks corresponding to the signature, require synchronization; and
marking, in a synchronization structure, a first set of one or more locations on the first de-duplicated physical storage on the production site and a second set of one or more locations on the second de-duplicated physical storage on the replication site, the first set of one or more locations corresponding to the first set of blocks and the second set of one or more locations corresponding to the second set of blocks, the first set of blocks and the second set of blocks corresponding to the signature that require synchronization.
10. The computer program product of claim 9 , wherein a splitter intercepts IOs for the virtual machine disk on the production site, and marks each location corresponding to the IOs arriving to the virtual machine disk on the production site as dirty in the synchronization structure.
11. The computer program product of claim 9 , wherein a first snapshot of a first LU containing a first file system containing the virtual machine disk on the production site is created on the production site and a second snapshot of a second LU containing a second file system containing the virtual machine disk on the replication site is created on the replication site, and where the first snapshot is attached to a first replication appliance at the production site and the second snapshot is attached to a second replication appliance at the replication site.
12. The computer program product of claim 11 , wherein block hashes are read at the production site from the first LU using de-duplicated storage APIs.
13. The computer program product of claim 12 , wherein the block hashes from the production site are used with an offset of the second VMDK by a write by Hash command.
14. The computer program product of claim 13 , wherein the offset of the second LU used by the write by Hash command is based on the second mapping between the VMDK and the physical LU on the replication site.
15. The computer program product of claim 14 , wherein if the write by Hash command fails, then each location associated with the failed write by Hash command is added to the synchronization structure for synchronization at a later point in time.
16. A system comprising:
a production site;
a replication site; and
computer-executable logic operating in memory, wherein the computer-executable program logic is configured to enable execution across one or more processors of:
reading a signature corresponding to a portion of data on a virtual machine disk (VMDK) on a production site, wherein reading the signature includes obtaining a first mapping between a first block offset on the virtual machine disk on the production site and a second block offset on a first de-duplicated physical storage on the production site, and reading, based on the first mapping, the signature from the first de-duplicated physical storage on the production site;
sending the signature to a replication site;
obtaining a second mapping between a third block offset on a virtual machine disk on the replication site and a fourth block offset on a second de-duplicated physical disk storage on the replication site;
issuing a command to write the portion of data corresponding to the signature to the second de-duplicated physical storage on the replication site based on the second mapping;
determining, based at least in part on a result of the command, that a first set of blocks on the first de-duplicated physical storage on the production site and a second set of blocks on the second de-duplicated physical storage on the replication site, the first set of blocks and the second set of blocks corresponding to the signature, require synchronization; and
marking, in a synchronization structure, a first set of one or more locations on the first de-duplicated physical storage on the production site and a second set of one or more locations on the second de-duplicated physical storage on the replication site, the first set of one or more locations corresponding to the first set of blocks and the second set of one or more locations corresponding to the second set of blocks, the first set of blocks and the second set of blocks corresponding to the signature that require synchronization.
17. The system of claim 16 , wherein a splitter intercepts IOs for the virtual machine disk on the production site, and marks each location corresponding to the IOs arriving to the virtual machine disk on the production site as dirty in the synchronization structure.
18. The system of claim 16 , wherein a first snapshot of a first LU containing a first file system containing the virtual machine disk on the production site is created on the production site and a second snapshot of a second LU containing a second file system containing the virtual machine disk on the replication site is created on the replica site, and where the first snapshot is attached to a first replication appliance at the production site and the second snapshot is attached to a second replication appliance at the replication site.
19. The system of claim 18 , wherein block hashes are read at the production site from the first LU using de-duplicated storage APIs, and wherein the block hashes from the production site are used with an offset of the second VMDK by a write by Hash command, and wherein the offset of the second LU used by the write by Hash command is based on the second mapping between the VMDK and the physical LU on the replication site, and wherein if the write by Hash command fails, then each location associated with the failed write by Hash command is added to the synchronization structure for synchronization at a later point in time.