Data migration between multiple tiers in a storage system using pivot tables
View Patent ↗Technologies are described for implementing a migration mechanism in a data storage system containing multiple tiers of storage with each tier having different cost and performance parameters. Access statistics can be collected for each territory, or storage entity, within the storage system. Data that is accessed more frequently can be migrated toward higher performance storage tiers while data that is accessed less frequently can be migrated towards lower performance storage tiers. Each tier can be associated with a range of ILM statistics referred to as the bucket for that tier. A pivot table may be provided that relates the tiers and the buckets. Operations on the pivot table can provide counts of how many territories may be promoted or demoted between any two pairs of tiers.
1. A method for migrating data between tiers of a storage system, the method comprising:
subdividing a storage capacity of the storage system into a set of territories;
partitioning the set of territories into tiers;
partitioning a range of information lifecycle management metrics into buckets, each bucket corresponding to one of the tiers;
maintaining a data structure relating the buckets and the tiers, the data structure containing an entry for each relationship of a bucket and a tier that is a number of territories currently stored in the tier having an information lifecycle management number within the range of the bucket, wherein the entry corresponds to a count of territories stored in the tier and having a lifecycle management metric within the bucket; and
moving a territory from a current tier to destination tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the destination tier.
2. The method of claim 1 , wherein moving a tier comprises promoting a territory from a lower tier to a higher tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the higher tier.
3. The method of claim 1 , wherein moving a tier comprises demoting a territory from a higher tier to a lower tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the lower tier.
4. The method of claim 1 , further comprising assigning each bucket and each tier a value in a hierarchical order, wherein territories in a bucket and a tier having a same value are considered to be properly placed.
5. The method of claim 4 , wherein territories in a bucket and a tier having a different value are considered to be candidates for moving.
6. The method of claim 1 , wherein partitioning the range of all possible information lifecycle management metrics comprises establishing a partition based on a physical capacity of each tier, and a cumulative statistical distribution of territories with respect to information lifecycle management metrics.
7. The method of claim 6 , wherein partitioning the range of all possible information lifecycle management metrics comprises providing slack space within each tier to be used during data migrations.
8. A method for migrating data between tiers of a storage system, the method comprising:
subdividing a storage capacity of the storage system into a set of territories;
partitioning the set of territories into tiers;
partitioning a range of information lifecycle management metrics into buckets, each bucket corresponding to one of the tiers;
maintaining a data structure relating the buckets and the tiers, the data structure containing an entry for each relationship of a bucket and a tier that is a number of territories currently stored in the tier having an information lifecycle management number within the range of the bucket; and
moving a territory from a current tier to destination tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the destination tier, wherein maintaining an information lifecycle management metric comprises combining an age statistic and a frequency statistic into a single metric.
9. A computer storage medium having computer-executable instructions stored thereon which, when executed by a computer system, cause the computer system to:
subdivide a storage capacity of a storage system into a set of territories;
partition the set of territories into tiers;
partition a range of information lifecycle management metrics into buckets, each bucket corresponding to one of the tiers;
maintain a data structure relating the buckets and the tiers, the data structure containing an entry for each relationship of a bucket and a tier that is a number of territories currently stored in the tier having an information lifecycle management number within the range of the bucket;
move a territory from a current tier to destination tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the destination tier;
promote a territory from a lower tier to a higher tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the higher tier; and
determine how many territories to promote into a destination tier based upon a count of territories in all lower tiers having a lifecycle management metric within the bucket corresponding to the destination tier.
10. The computer storage medium of claim 9 , wherein partitioning the range of all possible information lifecycle management metrics comprises providing a periodic timer for updating the partitioning.
11. The computer storage medium of claim 9 , wherein partitioning the range of all possible information lifecycle management metrics comprises establishing a partition based on a physical capacity of each tier, and a cumulative statistical distribution of territories with respect to information lifecycle management metrics.
12. A computer storage medium having computer-executable instructions stored thereon which, when executed by a computer system, cause the computer system to:
subdivide a storage capacity of a storage system into a set of territories;
partition the set of territories into tiers;
partition a range of information lifecycle management metrics into buckets, each bucket corresponding to one of the tiers;
maintain a data structure relating the buckets and the tiers, the data structure containing an entry for each relationship of a bucket and a tier that is a number of territories currently stored in the tier having an information lifecycle management number within the range of the bucket;
move a territory from a current tier to destination tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the destination tier;
demote a territory from a higher tier to a lower tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the lower tier; and
determine how many territories to demote from a source tier into a destination tier based upon a count of territories in the source tier having a lifecycle management metric within the bucket corresponding to the destination tier.
13. A computer storage medium having computer-executable instructions stored thereon which, when executed by a computer system, cause the computer system to:
subdivide a storage capacity of a storage system into a set of territories;
partition the set of territories into tiers;
partition a range of information lifecycle management metrics into buckets, each bucket corresponding to one of the tiers;
maintain a data structure relating the buckets and the tiers, the data structure containing an entry for each relationship of a bucket and a tier that is a number of territories currently stored in the tier having an information lifecycle management number within the range of the bucket; and
move a territory from a current tier to destination tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the destination tier, wherein maintaining an information lifecycle management metric comprises combining an age statistic and a frequency statistic into a single metric.
14. A data storage system comprising:
a storage subsystem;
a plurality of physical storage devices associated with the storage subsystem; and
one or more data migration modules operable to cause the storage subsystem to:
subdivide a storage capacity of the storage system into a set of territories;
partition the set of territories into tiers;
partition a range of information lifecycle management metrics into buckets, each bucket corresponding to one of the tiers;
maintain a data structure relating the buckets and the tiers, the data structure containing an entry for each relationship of a bucket and a tier that is a number of territories currently stored in the tier having an information lifecycle management number within the range of the bucket, wherein the entry corresponds to a count of territories stored in the tier and having a lifecycle management metric within the bucket; and
move a territory from a current tier to destination tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the destination tier.
15. The data storage system of claim 14 , wherein the data migration modules are further operable to cause the storage subsystem to promote a territory from a lower tier to a higher tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the higher tier.
16. The data storage system of claim 14 , wherein the data migration modules are further operable to cause the storage subsystem to demote a territory from a higher tier to a lower tier in response to the information lifecycle management metric associated with the territory being within the bucket associated with the lower tier.