Duplication between disparate deduplication systems
Disclosed herein are methods, systems, and processes to improve the duplication of data between disparate deduplication systems. Source fingerprints are generated for data blocks using a source fingerprint algorithm at a source deduplication system. The source fingerprints and previously-generated source fingerprints are used to determine whether the data blocks are new or modified. If the data blocks are new or modified, target fingerprints are generated for the data blocks using a target fingerprint algorithm associated with a target deduplication system. The target fingerprints are sent to the target deduplication system.
1. A method comprising:
generating, at a source deduplication system, a source fingerprint for each of one or more data blocks using a source fingerprint algorithm;
determining whether the one or more data blocks are new or modified using the source fingerprint generated for each of the one or more data blocks and one or more previously-generated source fingerprints; and
if a determination is made that the one or more of the one or more data blocks are new or modified,
generating a target fingerprint for each of the one or more of the one or more data blocks using a target fingerprint algorithm associated with a target deduplication system, and
sending the target fingerprint for each of the one or more of the one or more data blocks to the target deduplication system.
2. The method of claim 1 , further comprising:
receiving a request from the target deduplication system for the one or more of the one or more data blocks that are new or modified; and
based on the request, sending the one or more of the one or more data blocks that are new or modified to the target deduplication system.
3. The method of claim 2 , further comprising:
determining a reference count of each of the one or more data blocks; and
sending the reference count of each of the one or more data blocks to the target deduplication system as part of sending the one or more of the one or more data blocks that are new or modified.
4. The method of claim 2 , further comprising:
converting the source fingerprint for each of the one or more of the one or more data blocks that are new or modified to the target fingerprint generated using the target fingerprint algorithm associated with the target deduplication system.
5. The method of claim 1 , wherein
the one or more previously-generated source fingerprints are generated using the source fingerprint algorithm, and
the source fingerprint algorithm and the target fingerprint algorithm are different from one another.
6. The method of claim 1 , further comprising:
accessing the target fingerprint algorithm associated with the target deduplication system from a computing system communicatively coupled to the source deduplication system, or
receiving the target fingerprint algorithm associated with the target deduplication system from the target deduplication system.
7. The method of claim 1 , wherein
the one or more data blocks are deduplicated by the source deduplication system prior to the generation of the target fingerprint for each of the one or more of the one or more data blocks.
8. The method of claim 1 , further comprising:
determining whether the one or more data blocks have been previously sent to the target deduplication system.
9. A non-transitory computer readable storage medium comprising program instructions executable to:
generate, at a source deduplication system, a source fingerprint for each of one or more data blocks using a source fingerprint algorithm;
determine whether the one or more data blocks are new or modified using the source fingerprint generated for each of the one or more data blocks and one or more previously-generated source fingerprints; and
if a determination is made that the one or more of the one or more data blocks are new or modified,
generate a target fingerprint for each of the one or more of the one or more data blocks using a target fingerprint algorithm associated with a target deduplication system, and
send the target fingerprint for each of the one or more of the one or more data blocks to the target deduplication system.
10. The non-transitory computer readable storage medium of claim 9 , further comprising:
receiving a request from the target deduplication system for the one or more of the one or more data blocks that are new or modified; and
based on the request, sending the one or more of the one or more data blocks that are new or modified to the target deduplication system.
11. The non-transitory computer readable storage medium of claim 10 , further comprising:
determining a reference count of each of the one or more data blocks; and
sending the reference count of each of the one or more data blocks to the target deduplication system as part of sending the one or more of the one or more data blocks that are new or modified.
12. The non-transitory computer readable storage medium of claim 9 , wherein
the one or more previously-generated source fingerprints are generated using the source fingerprint algorithm, and
the source fingerprint algorithm and the target fingerprint algorithm are different from one another.
13. The non-transitory computer readable storage medium of claim 9 , further comprising:
determining whether the one or more data blocks have been previously sent to the target deduplication system.
14. The non-transitory computer readable storage medium of claim 9 , wherein
the one or more data blocks are deduplicated by the source deduplication system prior to the generation of the target fingerprint for each of the one or more of the one or more data blocks.
15. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, wherein the memory stores program instructions executable by the one or more processors to:
generate, at a source deduplication system, a source fingerprint for each of one or more data blocks using a source fingerprint algorithm;
determine whether the one or more data blocks are new or modified using the source fingerprint generated for each of the one or more data blocks and one or more previously-generated source fingerprints; and
if a determination is made that the one or more of the one or more data blocks are new or modified,
generate a target fingerprint for each of the one or more of the one or more data blocks using a target fingerprint algorithm associated with a target deduplication system, and
send the target fingerprint for each of the one or more of the one or more data blocks to the target deduplication system.
16. The system of claim 15 , further comprising:
receiving a request from the target deduplication system for the one or more of the one or more data blocks that are new or modified; and
based on the request, sending the one or more of the one or more data blocks that are new or modified to the target deduplication system.
17. The system of claim 16 , further comprising:
determining a reference count of each of the one or more data blocks; and
sending the reference count of each of the one or more data blocks to the target deduplication system as part of sending the one or more of the one or more data blocks that are new or modified.
18. The system of claim 15 , wherein
the one or more previously-generated source fingerprints are generated using the source fingerprint algorithm, and
the source fingerprint algorithm and the target fingerprint algorithm are different from one another.
19. The system of claim 15 , further comprising:
determining whether the one or more data blocks have been previously sent to the target deduplication system.
20. The system of claim 15 , wherein
the one or more data blocks are deduplicated by the source deduplication system prior to the generation of the target fingerprint for each of the one or more of the one or more data blocks.