Sharing services between deduplication systems
Disclosed herein are methods, systems, and processes to share data storage-related services between multiple deduplication systems. In one embodiment, the method comprises receiving an indication that a local file corresponds to a shared file; virtually segmenting the shared file into a plurality of data segments, where virtually segmenting the shared file generates data segment fingerprints and library virtual mapping metadata, and the library virtual mapping metadata comprises fingerprint information corresponding to each of the plurality of data segments, and at least one of a unique file identifier, an offset of each data segment, and a size of each data segment; and transmitting the library virtual mapping metadata, where the library virtual mapping metadata is configured to be used in a subsequent deduplication operation.
1. A computer-implemented method comprising:
receiving an indication that a local file corresponds to a shared file;
virtually segmenting the shared file into a plurality of data segments, wherein
the virtually segmenting the shared file generates data segment fingerprints and library virtual mapping metadata, and
the library virtual mapping metadata comprises
fingerprint information corresponding to each of the plurality of data segments, and
at least one of a unique file identifier, an offset of each data segment, and a size of each data segment; and
transmitting the library virtual mapping metadata, wherein
the library virtual mapping metadata is configured to be used in a subsequent deduplication operation.
2. The computer-implemented method of claim 1 , wherein
the shared file exists in a library of shared files, and
the library of shared files is maintained by a library server.
3. The computer-implemented method of claim 1 , wherein
the library virtual mapping metadata is transmitted from a library server to a deduplication system.
4. The computer-implemented method of claim 3 , wherein
the library virtual mapping metadata identifies the deduplication system, and
the library virtual mapping metadata permits identification of one or more data segments of the plurality of data segments of the local file that can be excluded from a backup and deduplication operation.
5. The computer-implemented method of claim 3 , further comprising:
transmitting, to the deduplication server, a name of a file fingerprinting algorithm used to generate the data segment fingerprints, wherein
the name of the file generating algorithm is configured to be used by the deduplication server to generate one or more local file identifiers corresponding to the local file.
6. The computer-implemented method of claim 1 , wherein
the library virtual mapping metadata is further configured to permit identification of one or more data segments of the plurality of data segments of the shared file that can be excluded from a subsequent backup and deduplication operation.
7. The computer-implemented method of claim 1 , wherein
the shared file is shared by a plurality of deduplication systems.
8. The computer-implemented method of claim 1 , further comprising:
storing the library virtual mapping metadata.
9. A non-transitory computer-readable storage medium comprising program instructions executable to perform a method comprising:
receiving an indication that a local file corresponds to a shared file;
virtually segmenting the shared file into a plurality of data segments, wherein
the virtually segmenting the shared file generates data segment fingerprints and library virtual mapping metadata, and
the library virtual mapping metadata comprises
fingerprint information corresponding to each of the plurality of data segments, and
at least one of a unique file identifier, an offset of each data segment, and a size of each data segment; and
transmitting the library virtual mapping metadata, wherein
the library virtual mapping metadata is configured to be used in a subsequent deduplication operation.
10. The non-transitory computer-readable storage medium of claim 9 , wherein
the shared file exists in a library of shared files, and
the library of shared files is maintained by a library server.
11. The non-transitory computer-readable storage medium of claim 9 , wherein
the library virtual mapping metadata is transmitted from a library server to a deduplication system.
12. The non-transitory computer-readable storage medium of claim 11 , wherein
the library virtual mapping metadata identifies the deduplication system, and
the library virtual mapping metadata permits identification of one or more data segments of the plurality of data segments of the local file that can be excluded from a backup and deduplication operation.
13. The non-transitory computer-readable storage medium of claim 11 , wherein the method further comprises:
transmitting, to the deduplication server, a name of a file fingerprinting algorithm used to generate the data segment fingerprints, wherein
the name of the file generating algorithm is configured to be used by the deduplication server to generate one or more local file identifiers corresponding to the local file.
14. The non-transitory computer-readable storage medium of claim 9 , wherein
the library virtual mapping metadata is further configured to permit identification of one or more data segments of the plurality of data segments of the shared file that can be excluded from a subsequent backup and deduplication operation.
15. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, wherein the memory stores program instructions executable by the one or more processors to perform a method comprising
receiving an indication that a local file corresponds to a shared file,
virtually segmenting the shared file into a plurality of data segments, wherein
the virtually segmenting the shared file generates data segment fingerprints and library virtual mapping metadata, and
the library virtual mapping metadata comprises
fingerprint information corresponding to each of the plurality of data segments, and
at least one of a unique file identifier, an offset of each data segment, and a size of each data segment, and
transmitting the library virtual mapping metadata, wherein
the library virtual mapping metadata is configured to be used in a subsequent deduplication operation.
16. The system of claim 15 , wherein
the shared file exists in a library of shared files, and
the library of shared files is maintained by a library server.
17. The system of claim 15 , wherein
the library virtual mapping metadata is transmitted from a library server to a deduplication system.
18. The system of claim 17 , wherein
the library virtual mapping metadata identifies the deduplication system, and
the library virtual mapping metadata permits identification of one or more data segments of the plurality of data segments of the local file that can be excluded from a backup and deduplication operation.
19. The system of claim 17 , wherein the method further comprises:
transmitting, to the deduplication server, a name of a file fingerprinting algorithm used to generate the data segment fingerprints, wherein
the name of the file generating algorithm is configured to be used by the deduplication server to generate one or more local file identifiers corresponding to the local file.
20. The system of claim 15 , wherein
the library virtual mapping metadata is further configured to permit identification of one or more data segments of the plurality of data segments of the shared file that can be excluded from a subsequent backup and deduplication operation.