System and method for intelligent data movements between non-deduplicated and deduplicated tiers in a primary storage array

1. A data storage system comprising:

a first storage tier including a first set of storage devices;

a second storage tier including a second set of storage devices, the second storage tier including deduplicated data; and

a quality of service (QoS) manager to manage data migration between the first storage tier and the second storage tier, the QoS manager to maintain deduplication statistics for the deduplicated data, the deduplication statistics determining a degree to which a data extent is to be migrated, and determined by performing a partial deduplication for the data extent and querying a deduplication fingerprint index, wherein performing the partial deduplication for the data extent includes computing fingerprints for data within the data extent and determining an amount of the data within the data extent, wherein the QoS manager, based on the deduplication statistics, is to partially migrate the data extent between the first storage tier and the second storage tier, the data extent including data stored at a range of storage addresses.

2. The data storage system as in claim 1 , additionally including a primary storage array including the first storage tier and the second storage tier, the primary storage array to provide primary storage to one or more clients.

3. The data storage system as in claim 2 , wherein the QoS manager is to compute a movement cost for the data extent, the movement cost including a cost in input/output (I/O) resources and computational resources to migrate the data extent between the first storage tier and the second storage tier.

4. The data storage system as in claim 3 , wherein the movement cost includes a cost to deduplicate or rehydrate the data extent based on the deduplication statistics.

5. The data storage system as in claim 1 , wherein to partially migrate the data extent, the QoS manager is to:

select a data extent to migrate from the first storage tier to the second storage tier based on I/O activity for the data extent;

determine a set of duplicate instances of the extent in the first storage tier;

store a single instance of the extent to the second storage tier; and

determine a subset of the set of duplicate instances of the extent in the first storage tier to remove from the first storage tier.

6. The data storage system as in claim 5 , wherein to partially migrate a data extent additionally includes to remove less than all duplicate instances of the extent from the first storage tier.

7. The data storage system as in claim 5 , wherein to select the data extent to migrate from the first storage tier to the second storage tier includes to determine that I/O activity for the data extent is below a threshold associated with the first storage tier.

8. The data storage system as in claim 1 , wherein to partially migrate the data extent, the QoS manager is to:

select a data extent to migrate from the second storage tier to the first storage tier based on I/O activity for the data extent;

determine a set of duplicate instances of the extent to be stored in the first storage tier; and

store less than an entire set of duplicate instances of the extent to the first storage tier.

9. The data storage system as in claim 8 , wherein the QoS manager is further to:

select a subset of the set of duplicate instances of the extent not migrated to the first storage tier; and

configure storage virtualization for the first storage tier to service I/O requests for the subset of the set of duplicate instances of the extent via the second storage tier.

10. The data storage system as in claim 8 , wherein to select the data extent to migrate from the second storage tier to the first storage tier includes to determine that I/O activity for the data extent is above a threshold associated with the second storage tier.

11. A method of managing a primary storage array including a deduplicated storage tier, the method comprising:

monitoring quality of service (QoS) statistics associated with data stored on the primary storage array;

determining, based on the QoS statistics, that a data extent stored on the primary storage array is a candidate for migration between tiers of the primary storage array;

determining a dedupability statistic by performing a partial deduplication for the data extent and querying a deduplication fingerprint index, wherein the dedupability statistic determines a degree to which the data extent is to be migrated, wherein performing the partial deduplication for the data extent includes computing fingerprints for data within the data extent and determining an amount of the data within the data extent;

computing a migration cost for the data extent based in part on the dedupability statistic for the data extent; and

performing a partial migration of the data extent when the migration cost is above a total migration threshold.

12. The method as in claim 11 , additionally comprising performing a total migration of the extent when the migration cost is below the total migration threshold.

13. The method as in claim 11 , wherein computing the migration cost of the storage extent includes:

determining that the storage extent is to be migrated from a non-deduplicated tier of the primary storage array to a deduplicated tier of the primary storage array;

determining an amount of processor resources to deduplicate the storage extent; and

determining an amount of I/O resources to migrate the storage extent after deduplication.

14. The method as in claim 11 , wherein computing the migration cost of the storage extent includes:

determining that the storage extent is to be migrated from a deduplicated tier of the primary storage array to a non-deduplicated tier of the primary storage array;

determining an amount of processor resources to rehydrate the storage extent; and

determining an amount of I/O resources to migrate the storage extent after rehydration.

15. The method as in claim 11 , wherein the dedupability statistic for the storage extent includes a deduplication ratio associated with the storage extent.

16. One or more non-transitory machine readable media storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations to manage a primary storage array, the operations including:

monitoring quality of service (QoS) statistics associated with data stored on the primary storage array;

determining, based on the QoS statistics, that a data extent stored on the primary storage array is a candidate for migration between tiers of the primary storage array;

determining a dedupability statistic by performing a partial deduplication for the data extent and querying a deduplication fingerprint index, wherein the dedupability statistic determines a degree to which the data extent is to be migrated, wherein performing the partial deduplication for the data extent includes computing fingerprints for data within the data extent and determining an amount of the data within the data extent;

computing a migration cost for the data extent based in part on the dedupability statistic for the data extent; and

performing a partial migration of the data extent when the migration cost is above a total migration threshold.

17. The one or more non-transitory machine readable media as in claim 16 , the operations additionally comprising performing a total migration of the extent when the migration cost is below the total migration threshold.

18. The one or more non-transitory machine readable media as in claim 16 , wherein computing the migration cost of the storage extent includes:

determining that the storage extent is to be migrated from a non-deduplicated tier of the primary storage array to a deduplicated tier of the primary storage array;

determining an amount of processor resources to deduplicate the storage extent; and

determining an amount of I/O resources to migrate the storage extent after deduplication.

19. The one or more non-transitory machine readable media as in claim 16 , wherein computing the migration cost of the storage extent includes:

determining that the storage extent is to be migrated from a deduplicated tier of the primary storage array to a non-deduplicated tier of the primary storage array;

determining an amount of processor resources to rehydrate the storage extent; and

determining an amount of I/O resources to migrate the storage extent after rehydration.

20. The one or more non-transitory machine readable media as in claim 16 , wherein the dedupability statistic for the storage extent includes a deduplication ratio associated with the storage extent.