Method and system for replication
Techniques are used for replication in a storage system. The techniques may be used to provide, among other things, the determination that a data block has more than one instance in a storage system. A similarity hash for the data block is determined, and an instruction is transmitted from a source system to a target system to replicate the data block based on the similarity hash.
1. A method for replication, comprising:
accessing, by a storage system, a reference count of a data block, wherein the reference count is accessed from metadata of the data block;
based on the accessing, determining, by the source system, that a data block has more than one instance in a storage system;
determining a similarity hash for the data block, wherein the similarity hash is based on a non-cryptographic hash function;
transmitting an instruction to a target system to replicate the data block based on the similarity hash;
receiving the instruction to replicate the data block based on the similarity hash and a cryptographic hash of the data block;
searching a deduplication table on the target system for the similarity hash;
verifying a data block from the deduplication table matches the data block corresponding to the received similarity hash;
computing a cryptographic hash of the data block from the deduplication table;
comparing the cryptographic hash of the data block from the deduplication table and the received cryptographic hash from the instruction; and
replicating the data block from the deduplication table.
2. The method of claim 1 , wherein determining that the data block has more than one instance in the storage system comprises:
determining that a reference count of the data block exceeds one.
3. The method of claim 1 , wherein determining that the data block has more than one instance in the storage system comprises:
accessing metadata of the data block.
4. The method of claim 1 , further comprising: determining a cryptographic hash of the data block.
5. The method of claim 4 , wherein transmitting the instruction to the target system to replicate the data block comprises:
transmitting the similarity hash and the cryptographic hash of the data block.
6. The method of claim 1 , further comprising: determining the data block has been previously transmitted to the target system.
7. A system for replication in a storage system, the system comprising at least one processor configured to:
access, by the storage system, a reference count of a data block, wherein the reference count is accessed from metadata of the data block;
based on the accessing, determine, by the source system, that a data block has more than one instance in a storage system;
determine a similarity hash for the data block, wherein the similarity hash is based on a non-cryptographic hash function;
transmit an instruction to a target system to replicate the data block based on the similarity hash;
receive the instruction to replicate the data block based on the similarity hash and a cryptographic hash of the data block;
search a deduplication table on the target system for the similarity hash;
verify a data block from the deduplication table matches the data block corresponding to the received similarity hash;
compute a cryptographic hash of the data block from the deduplication table;
compare the cryptographic hash of the data block from the deduplication table and the received cryptographic hash from the instruction; and
replicate the data block from the deduplication table.
8. The system of claim 7 , wherein the processor is further configured to: determine that a reference count of the data block exceeds one.
9. The system of claim 7 , wherein the processor is further configured to: access metadata of the data block.
10. The system of claim 7 , wherein the processor is further configured to: determine a cryptographic hash of the data block.
11. The system of claim 10 , wherein the processor is further configured to: transmit the similarity hash and the cryptographic hash of the data block.
12. The system of claim 7 , wherein the processor is further configured to: determine the data block has been previously transmitted to the target system.