Identification of high deduplication data

1. A computer-implemented method for identifying portions of a dataset with high deduplication potential, comprising:

determining a sample size of a plurality of chunks to be sampled for each region of a plurality of regions included in a dataset, wherein the sample size is determined based, at least in part, on:

a likelihood of identifying at least one identical hash value between a first region corresponding to a portion of a first system backup and a second region corresponding to a portion of a second system backup included in the dataset;

sampling the plurality of chunks for each region of the plurality of regions based on the determined sample size;

generating a hash value in an index for each chunk of the plurality of chunks sampled for each region of the plurality of regions;

comparing hash values calculated for each chunk of the plurality of chunks sampled for respective regions against hash values calculated for each chunk of the plurality of chunks sampled for all other regions of the plurality of regions;

determining that two regions include identical data based, at least in part, on two regions sharing one or more identical hash values;

selecting a first series of regions of the plurality of regions for migration to a first storage system that supports data deduplication based, at least in part, on each successive region in the first series of regions sharing a number of identical hash values above a given threshold with a previous region in the first series of regions; and

migrating the first series of regions to the first storage system.

2. The computer-implemented method of claim 1 , wherein the likelihood of identifying the at least one collision between the first region corresponding to the first system backup and the second region corresponding to the second system backup included in the dataset is based, at least in part, on:

identifying a number of system backups included in the dataset; and

determining a degree of similarity between the first system backup and the second system backup included in the dataset.

3. The computer-implemented method of claim 1 , wherein storing each hash value further comprises:

storing a first location of the region corresponding to the hash value in the index; and

storing a second location within the region corresponding to the hash value in the index.

4. The computer-implemented method of claim 1 , further comprising:

selecting a second series of regions of the plurality of regions for migration to the first storage system that supports data deduplication based, at least in part, on each successive region of a number of regions above a given threshold sharing at least one identical hash value with a previous region in the second series of regions; and

migrating the second series of regions to the first storage system.

5. The computer-implemented method of claim 1 , further comprising:

selecting a third series of regions of the plurality of regions for migration to the first storage system that supports data deduplication based, at least in part, on:

each successive region in the third series of regions sharing a number of identical hash values above a first given threshold with a previous region in the third series of regions; and

the number of regions included in the third series of regions exceeding a second given threshold; and

migrating the second series of regions to the first storage system.

6. A computer program product, the computer program product comprising one or more computer readable storage media and program instructions stored on said one or more computer readable storage media, said program instructions comprising instructions to:

determine a sample size of a plurality of chunks to be sampled for each region of a plurality of regions included in a dataset, wherein the sample size is determined based, at least in part, on:

a likelihood of identifying at least one identical hash value between a first region corresponding to a portion of a first system backup and a second region corresponding to a portion of a second system backup included in the dataset;

sample the plurality of chunks for each region of the plurality of regions based on the determined sample size;

generate a hash value in an index for each chunk of the plurality of chunks sampled for each region of the plurality of regions;

compare hash values calculated for each chunk of the plurality of chunks sampled for respective regions against hash values calculated for each chunk of the plurality of chunks sampled for all other regions of the plurality of regions;

determine that two regions include identical data based, at least in part, on two regions sharing one or more identical hash values;

select a first series of regions of the plurality of regions for migration to a first storage system that supports data deduplication based, at least in part, on each successive region in the first series of regions sharing a number of identical hash values above a given threshold with a previous region in the first series of regions; and

migrate the first series of regions to the first storage system.

7. The computer program product of claim 6 , wherein the likelihood of identifying the at least one collision between the first region corresponding to the first system backup and the second region corresponding to the second system backup included in the dataset is based, at least in part, on:

identifying a number of system backups included in the dataset; and

determining a degree of similarity between the first system backup and the second system backup included in the dataset.

8. The computer program product of claim 6 , wherein the program instructions to store each hash value further comprise instructions to:

store a first location of the region corresponding to the hash value in the index; and

store a second location within the region corresponding to the hash value in the index.

9. The computer program product of claim 6 , further comprising program instructions to:

select a second series of regions of the plurality of regions for migration to the first storage system that supports data deduplication based, at least in part, on each successive region of a number of regions above a given threshold sharing at least one identical hash value with a previous region in the second series of regions; and

migrate the second series of regions to the first storage system.

10. The computer program product of claim 6 , further comprising program instructions to:

select a third series of regions of the plurality of regions for migration to the first storage system that supports data deduplication based, at least in part, on:

each successive region in the third series of regions sharing a number of identical hash values above a first given threshold with a previous region in the third series of regions; and

the number of regions included in the third series of regions exceeding a second given threshold; and

migrate the second series of regions to the first storage system.

11. A computer system, the computer system comprising:

one or more computer processors;

one or more computer readable storage media;

computer program instructions;

said computer program instructions being stored on said one or more computer readable storage media for execution by said one or more computer processors; and

said computer program instructions comprising instructions to:

determine a sample size of a plurality of chunks to be sampled for each region of a plurality of regions included in a dataset, wherein the sample size is determined based, at least in part, on:

a likelihood of identifying at least one identical hash value between a first region corresponding to a portion of a first system backup and a second region corresponding to a portion of a second system backup included in the dataset;

sample the plurality of chunks for each region of the plurality of regions based on the determined sample size;

generate a hash value in an index for each chunk of the plurality of chunks sampled for each region of the plurality of regions;

compare hash values calculated for each chunk of the plurality of chunks sampled for respective regions against hash values calculated for each chunk of the plurality of chunks sampled for all other regions of the plurality of regions;

determine that two regions include identical data based, at least in part, on two regions sharing one or more identical hash values;

select a first series of regions of the plurality of regions for migration to a first storage system that supports data deduplication based, at least in part, on each successive region in the first series of regions sharing a number of identical hash values above a given threshold with a previous region in the first series of regions; and

migrate the first series of regions to the first storage system.

12. The computer system of claim 11 , wherein the likelihood of identifying the at least one collision between the first region corresponding to the first system backup and the second region corresponding to the second system backup included in the dataset is based, at least in part, on:

identifying a number of system backups included in the dataset; and

determining a degree of similarity between the first system backup and the second system backup included in the dataset.

13. The computer system of claim 11 , wherein the program instructions to store each hash value further comprise instructions to:

store a first location of the region corresponding to the hash value in the index; and

store a second location within the region corresponding to the hash value in the index.

14. The computer system of claim 11 , further comprising program instructions to:

select a second series of regions of the plurality of regions for migration to the first storage system that supports data deduplication based, at least in part, on each successive region of a number regions above a given threshold sharing at least one identical hash value with a previous region in the second series of regions; and

migrate the second series of regions to the first storage system.

15. The computer system of claim 11 , further comprising program instructions to:

select a third series of regions of the plurality of regions for migration to the first storage system that supports data deduplication based, at least in part, on:

each successive region in the third series of regions sharing a number of identical hash values above a first given threshold with each previous region in the third series of regions; and

the number of regions included in the third series of regions exceeding a second given threshold; and

migrate the third series of regions to the first storage system.