Fingerprint change during data operations
Various systems, methods, and processes for caching and referencing multiple fingerprints while data operations are ongoing are disclosed. A first fingerprint is generated based on a first fingerprinting process. The first fingerprint is stored in association with a second fingerprint, which is based on a second fingerprinting process. The first fingerprint and the second fingerprint are associated with the same data segment. Data operations such as a backup operation, a restore operation, or a replication operation can be performed while the conversion of the data segment from the second fingerprint to the first fingerprint is ongoing.
1. A computer implemented method comprising:
generating a first fingerprint, wherein
the first fingerprint is generated from a data segment using a first fingerprinting process; and
storing the first fingerprint in an index, wherein
the first fingerprint is stored in association with a second fingerprint stored in the index,
the second fingerprint was generated from the data segment using a second fingerprinting process,
the first fingerprint and the second fingerprint are associated with the data segment by virtue of being stored in the index with information identifying the data segment,
the index comprises at least one existing fingerprint that was generated from another data segment using the second fingerprinting process, and
the at least one existing fingerprint is not associated with another fingerprint generated using the first fingerprinting process.
2. The computer implemented method of claim 1 further comprising:
recalculating the second fingerprint based on the first fingerprinting process, wherein
the second fingerprint is previously calculated based on the second fingerprinting process, and
the second fingerprint is stored in the index and in a cache; and
upon recalculating the second fingerprint based on the first fingerprinting process,
storing the first fingerprint and the second fingerprint in the index and in the cache.
3. The computer implemented method of claim 2 , wherein
the first fingerprint is stored along with the second fingerprint locally in the index and remotely in the cache.
4. The computer implemented method of claim 1 , further comprising:
retrieving a data object comprising the second fingerprint of a last full backup, wherein
the data object is retrieved from a container file, and
the second fingerprint is stored locally in the container file comprising the index.
5. The computer implemented method of claim 4 , further comprising:
querying the data segment against the last full backup to determine whether a data item associated with the data segment is also associated with the second fingerprint.
6. The computer implemented method of claim 5 , further comprising:
querying the cache to determining whether the data item associated with the data segment is also associated with the second fingerprint, wherein
the second fingerprint is stored in the cache, and
querying the data segment against the last full backup fails to indicate that the data item associated with the data segment is also associated with the second fingerprint.
7. The computer implemented method of claim 6 , wherein
a new backup operation references the second fingerprint stored locally in the index and remotely in the cache.
8. The computer implemented method of claim 7 , further comprising:
clearing the cache after the new backup operation.
9. The computer implemented method of claim 2 , further comprising:
modifying the index based on the recalculating, wherein
the modification stores the first fingerprint associated with the data segment calculated with the first fingerprinting process along with the second fingerprint associated with the same data segment in the index; and
starting a conversion of data segments associated with the second fingerprint to first fingerprints based on the first fingerprinting process, wherein
the conversion is performed by accessing the modified index.
10. The computer implemented method of claim 9 , further comprising:
performing an another backup operation, a restore operation, or a replication operation while the conversion of the data segments is ongoing.
11. A non-transitory computer readable storage medium comprising program instructions executable to:
generate a first fingerprint, wherein
the first fingerprint is generated from a data segment using a first fingerprinting process; and
store the first fingerprint in an index, wherein
the first fingerprint is stored in association with a second fingerprint stored in the index,
the second fingerprint was generated from the data segment using a second fingerprinting process,
the first fingerprint and the second fingerprint are associated with the data segment by virtue of being stored in the index with information identifying the data segment,
the index comprises at least one existing fingerprint that was generated from another data segment using the second fingerprinting process, and
the at least one existing fingerprint is not associated with another fingerprint generated using the first fingerprinting process.
12. The non-transitory computer readable storage medium of claim 11 , further comprising:
recalculating the second fingerprint based on the first fingerprinting process, wherein
the second fingerprint is previously calculated based on the second fingerprinting process, and
the second fingerprint is stored in the index and in a cache; and
upon recalculating the second fingerprint based on the first fingerprinting process, storing the first fingerprint and the second fingerprint in the index and in the cache, wherein
the first fingerprint is stored along with the second fingerprint locally in the index and remotely in the cache.
13. The non-transitory computer readable storage medium of claim 11 , further comprising:
retrieving a data object comprising the second fingerprint of a last full backup, wherein the data object is retrieved from a container file, and
the second fingerprint is stored locally in the container file comprising the index.
14. The non-transitory computer readable storage medium of claim 13 , further comprising:
querying the data segment against the last full backup to determine whether a data item associated with the data segment is also associated with the second fingerprint; and
querying the cache to determining whether the data item associated with the data segment is also associated with the second fingerprint, wherein
the second fingerprint is stored in the cache, and
querying the data segment against the last full backup fails to indicate that the data item associated with the data segment is also associated with the second fingerprint.
15. The non-transitory computer readable storage medium of claim 12 , further comprising:
modifying the index based on the recalculating, wherein
the modification stores the first fingerprint associated with the data segment calculated with the first fingerprinting process along with the second fingerprint associated with the same data segment in the index;
starting a conversion of data segments associated with the second fingerprint to first fingerprints based on the first fingerprinting process, wherein
the conversion is performed by accessing the modified index; and
performing an another backup operation, a restore operation, or a replication operation while the conversion of the data segments is ongoing.
16. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, wherein the memory stores program instructions executable by the one or more processors to:
generate a first fingerprint, wherein
the first fingerprint is generated from a data segment using a first fingerprinting process; and
store the first fingerprint in an index, wherein
the first fingerprint is stored in association with a second fingerprint stored in the index,
the second fingerprint was generated from the data segment using a second fingerprinting process,
the first fingerprint and the second fingerprint are associated with the data segment by virtue of being stored in the index with information identifying the data segment,
the index comprises at least one existing fingerprint that was generated from another data segment using the second fingerprinting process, and
the at least one existing fingerprint is not associated with another fingerprint generated using the first fingerprinting process.
17. The system of claim 16 , further comprising:
recalculating the second fingerprint based on the first fingerprinting process, wherein
the second fingerprint is previously calculated based on the second fingerprinting process, and
the second fingerprint is stored in the index and in a cache; and
upon recalculating the second fingerprint based on the first fingerprinting process, storing the first fingerprint and the second fingerprint in the index and in the cache, wherein
the first fingerprint is stored along with the second fingerprint locally in the index and remotely in the cache.
18. The system of claim 16 , further comprising:
retrieving a data object comprising the second fingerprint of a last full backup, wherein
the data object is retrieved from a container file, and
the second fingerprint is stored locally in the container file comprising the index.
19. The system of claim 18 , further comprising:
querying the data segment against the last full backup to determine whether a data item associated with the data segment is also associated with the second fingerprint; and
querying the cache to determining whether the data item associated with the data segment is also associated with the second fingerprint, wherein
the second fingerprint is stored in the cache, and
querying the data segment against the last full backup fails to indicate that the data item associated with the data segment is also associated with the second fingerprint.
20. The system of claim 17 , further comprising:
modifying the index based on the recalculating, wherein
the modification stores the first fingerprint associated with the data segment calculated with the first fingerprinting process along with the second fingerprint associated with the same data segment in the index;
starting a conversion of data segments associated with the second fingerprint to first fingerprints based on the first fingerprinting process, wherein
the conversion is performed by accessing the modified index; and
performing an another backup operation, a restore operation, or a replication operation while the conversion of the data segments is ongoing.