Signature generator for use in comparing sets of data in a content addressable storage system
An apparatus in one embodiment comprises a storage system having a plurality of storage devices configured to store at least a first set of data. The storage system is configured to generate a first signature for the first set of data and to verify equivalence between the first set of data and a second set of data by comparing the first signature to a second signature generated for the second set of data. The first and second sets of data comprise respective first and second sets of pages with each page having a content-based page identifier and a page address, and the first and second signatures are generated as respective functions of the page identifiers and page addresses of the respective first and second sets of pages. The first and second sets of data may comprise respective first and second storage volumes, respective first and second sets of multiple storage volumes, or other sets of data.
1. An apparatus comprising:
a storage system comprising a plurality of storage devices;
the plurality of storage devices being configured to store at least a first set of data;
the storage system being configured to generate a first signature for the first set of data and to verify equivalence between the first set of data and a second set of data by comparing the first signature to a second signature generated for the second set of data;
wherein the first and second sets of data comprise respective first and second sets of pages with each page having a content-based page identifier and a page address;
wherein the first and second signatures are generated utilizing respective functions of the page identifiers and page addresses of the respective first and second sets of pages;
the first signature being determined at least in part by applying a combination of the page identifiers and the page addresses of each of the pages in the first set as input to a first function; and
the second signature being determined at least in part by applying a combination of the page identifiers and the page addresses of each of the pages in the second set as input to a second function.
2. The apparatus of claim 1 wherein the first and second sets of data comprise one of:
respective first and second storage volumes; and
respective first and second sets of multiple storage volumes.
3. The apparatus of claim 1 wherein the second set of data is stored in one of:
the plurality of storage devices of the storage system; and
a plurality of storage devices of a different storage system.
4. The apparatus of claim 1 wherein the storage system comprises a storage system of a target site and the first set of data comprises data replicated from a different storage system of a source site in accordance with a cycle-based asynchronous replication process and wherein the signatures of the first and second sets of data are compared as part of a data verification phase of at least one of the cycles of the cycle-based asynchronous replication process.
5. The apparatus of claim 1 wherein the storage system comprises a storage system of a target site and the first set of data comprises data copied from a different storage system of a source site in accordance with a migration process.
6. The apparatus of claim 1 wherein the page identifiers are generated as respective cryptographic hash functions of the data in the respective corresponding pages.
7. The apparatus of claim 1 wherein the storage system is configured:
to detect pages in the first set of data that are all-zero pages; and
to generate the first signature for the first set of data without utilizing the detected all-zero pages.
8. The apparatus of claim 1 wherein the storage system is configured:
to group consecutive pages of the first set of data into page groups;
to arrange the page groups into slices; and
to assign the slices to different ones of a plurality of slice processing units of the storage system.
9. The apparatus of claim 8 wherein each of the slice processing units is configured:
to generate slice signatures for each of the slices of the first set of data that are assigned to that slice processing unit; and
to combine the slice signatures using a first commutative function.
10. The apparatus of claim 9 wherein the storage system is configured to generate the first signature for the first set of data by combining the combined slice signatures from the multiple slice processing units using a second commutative function.
11. The apparatus of claim 10 wherein at least one of the first and second commutative functions comprises at least one of an XOR function and a modulo addition function.
12. The apparatus of claim 9 wherein a given one of the slice signatures is generated as a cryptographic hash function of an identifier of the first set of data and a plurality of page identifier and page address pairs of the corresponding slice.
13. The apparatus of claim 12 wherein the identifier of the first set of data comprises a storage volume identifier.
14. The apparatus of claim 12 wherein the cryptographic hash function comprises a cyclic redundancy check function.
15. A method comprising:
generating a first signature for a first set of data; and
verifying equivalence between the first set of data and a second set of data by comparing the first signature to a second signature generated for the second set of data;
wherein the first and second sets of data comprise respective first and second sets of pages with each page having a content-based page identifier and a page address;
wherein the first and second signatures are generated utilizing respective functions of the page identifiers and page addresses of the respective first and second sets of pages;
the first signature being determined at least in part by applying a combination of the page identifiers and the page addresses of each of the pages in the first set as input to a first function;
the second signature being determined at least in part by applying a combination of the page identifiers and the page addresses of each of the pages in the second set as input to a second function; and
wherein the method is implemented in a storage system comprising a plurality of storage devices configured to store at least the first set of data.
16. The method of claim 15 further comprising:
grouping consecutive pages of the first set of data into page groups;
arranging the page groups into slices; and
assigning the slices to different ones of a plurality of slice processing units of the storage system;
wherein each of the slice processing units is configured:
to generate slice signatures for each of the slices of the first set of data that are assigned to that slice processing unit; and
to combine the slice signatures using a first commutative function.
17. The method of claim 16 further comprising generating the first signature for the first set of data by combining the combined slice signatures from the multiple slice processing units using a second commutative function.
18. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by a storage system comprising a plurality of storage devices causes said storage system:
to generate a first signature for a first set of data stored in the plurality of storage devices; and
to verify equivalence between the first set of data and a second set of data by comparing the first signature to a second signature generated for the second set of data;
wherein the first and second sets of data comprise respective first and second sets of pages with each page having a content-based page identifier and a page address; and
wherein the first and second signatures are generated utilizing respective functions of the page identifiers and page addresses of the respective first and second sets of pages;
the first signature being determined at least in part by applying a combination of the page identifiers and the page addresses of each of the pages in the first set as input to a first function; and
the second signature being determined at least in part by applying a combination of the page identifiers and the page addresses of each of the pages in the second set as input to a second function.
19. The computer program product of claim 18 wherein the program code when executed by the storage system further causes said storage system:
to group consecutive pages of the first set of data into page groups;
to arrange the page groups into slices; and
to assign the slices to different ones of a plurality of slice processing units of the storage system;
wherein each of the slice processing units is configured:
to generate slice signatures for each of the slices of the first set of data that are assigned to that slice processing unit; and
to combine the slice signatures using a first commutative function.
20. The computer program product of claim 19 wherein the program code when executed by the storage system further causes said storage system to generate the first signature for the first set of data by combining the combined slice signatures from the multiple slice processing units using a second commutative function.