Data fingerprinting for copy accuracy assurance
View Patent ↗Systems and methods are disclosed for efficiently creating a data fingerprint to identify or characterize contents of a data object by using a selection function to select a plurality of non-contiguous regions from the data object, the selected regions each having a small number of bytes relative to the number of bytes in the data object and being distributed throughout the data object so that the selected regions comprise a sparse subset of the data of the data object yet provide a significant probability of including bytes that change if the data object were modified; and performing a hash operation on the data to produce a fingerprint based on the sparse subset of the data object. The data fingerprint thereby efficiently provides an indication of the contents of the data object, so that comparing data fingerprints can determine if the data objects are different if the corresponding fingerprints are different.
1. A method of creating a data fingerprint to identify or characterize contents of a data object using a selected subset of the data object, the method comprising:
using a selection function to select a plurality of non-contiguous regions from the data object,
the selection function using information about the contents of the data object to select the non-contiguous regions, wherein the selection function:
selects one or more portions of the data object with a high frequency of change among generations of the data object for inclusion in the selected regions; and
omits one or more portions of the data object with a low frequency of change among generations of the data object from inclusion in the selected regions;
the selected regions each having a small number of bytes relative to the number of bytes in the data object and being distributed throughout the data object, the selected regions comprising a sparse subset of the data of the data object and having a probability of including bytes that change if the data object were modified, the probability being based on a total size of the selected regions and a size of the data object; and
performing a hash operation on the data contained in the plurality of regions to produce a fingerprint based on the sparse subset of the data object.
2. The method of claim 1 , further comprising:
performing a binary hash operation on each region of the plurality of regions to create a plurality of hash values; and
combining the plurality of hash values into a unitary hash value.
3. The method of claim 1 , wherein the selection function is a parametrized function.
4. The method of claim 3 , wherein the parametrized function has a start value and a period value as parameters.
5. The method of claim 1 , further comprising selecting subsequent byte regions from the data object using an offset and a period by repeatedly adding the period to an offset of a previous sequence until the end of the data object is reached or exceeded.
6. The method of claim 1 , further comprising performing the hash operation on the combination of each byte region and an offset.
7. The method of claim 1 , wherein the selection function further omits one or more portions of the data object with unused space from inclusion in the selected regions.
8. The method of claim 4 , wherein:
the start value comprises a number representing an initial offset within the data object to begin the selection of the non-contiguous regions; and
the period value comprises a number representing a distance between offsets within the data object.
9. The method of claim 1 , further comprising:
comparing the fingerprint to a second fingerprint for a second data object; and
determining the data object and the second data object are different if the fingerprint and second fingerprint are different.