Successive data fingerprinting for copy accuracy assurance
View Patent ↗Systems and methods are disclosed for checking the data integrity of a data object copied between storage pools by comparing data fingerprints of data objects, comprising scheduling a series of successive copy operations over time for copying a data object from a source data store to a target data store; generating a partial fingerprint of the data object at the source data store that creates a fingerprint from a subset of the data object; sending the partial fingerprint of the data object to the target data store; sending any new data contents to the target data store; and creating a partial fingerprint of the data object at the target data store and comparing it to the received partial fingerprint to determine if they differ, thereby allowing incremental verification that the copy of the data object at the target data store is the same as at the source data store.
1. A method for checking data integrity of a data object copied between storage pools in a storage system by comparing partial fingerprints of data objects, the method comprising:
scheduling a series of successive copy operations over time for copying a data object from a source data store to a target data store, each of the successive copy operations being configured to copy a version of the data object at a different time than the other successive copy operations;
generating a source partial fingerprint of the data object for a copy operation from the successive copy operations at the source data store using a data fingerprinting operation that creates a fingerprint from a subset of data of the data object, the subset being selected by a dynamic function that selects different subsets in response to an initialization parameter that changes with each successive copy operation and represents an offset or location within the data object so that the data fingerprinting operation samples different portions throughout the data object based on the initialization parameter to determine the subset of data for the fingerprinting operation;
sending the source partial fingerprint of the data object to the target data store;
sending new data contents for the data object to the target data store, wherein the new data contents are not in an earlier version of the data object sent to the target data store for an earlier copy operation of the successive copy operations; and
creating a target partial fingerprint of the data object at the target data store and comparing it to the source partial fingerprint sent to the target data store to determine if they differ and thereby indicate that the data object at the target data store differs from the corresponding data object at the source data store,
thereby allowing incremental verification that the copy of the data object at the target data store is the same as at the source data store.
2. The method of claim 1 , wherein the data object comprises different versions of the same data object over time and wherein new data contents are sent as difference objects specifying the different data in a later version of a data object.
3. The method of claim 1 , wherein a copy operation will cause the generation and sending of a new source partial fingerprint but will not send any new data contents if the data object was unchanged from a prior copy operation.
4. The method of claim 1 , further comprising the data fingerprinting operation being configured to operate on a sparse subset of bits that is small relative to the size of each data object.
5. The method of claim 1 , further comprising the data fingerprinting operation selecting a subset of bits that changes from one fingerprint to the next.
6. The method of claim 1 , further comprising requesting for the target partial fingerprint to be generated at the target data store.