Data migration between multiple tiers in a storage system using pivot tables
View Patent ↗Technologies are described for implementing a migration mechanism in a data storage system containing multiple tiers of storage with each tier having different cost and performance parameters. Access statistics can be collected for each territory, or storage entity, within the storage system. Data that is accessed more frequently can be migrated toward higher performance storage tiers while data that is accessed less frequently can be migrated towards lower performance storage tiers. Each tier can be associated with a range of ILM statistics referred to as the bucket for that tier. A pivot table may be provided that relates the tiers and the buckets. Operations on the pivot table can provide counts of how many territories may be promoted or demoted between any two pairs of tiers.
1. A method for migrating data between tiers of a storage system, the method comprising:
subdividing a storage capacity of the storage system into a set of territories;
partitioning the set of territories into tiers;
partitioning a range of information lifecycle management metrics into buckets, each bucket corresponding to one of the tiers;
maintaining a data structure relating the buckets and the tiers, the data structure containing an entry for each relationship of a bucket and a tier that is a number of territories currently stored in the tier having an information lifecycle management metric within the range of the bucket;
moving a territory from a current tier to a destination tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the destination tier, wherein moving a territory comprises demoting a territory from a higher tier to a lower tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the lower tier.
2. The method of claim 1 , wherein moving a territory comprises promoting a territory from a lower tier to a higher tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the higher tier.
3. The method of claim 1 , wherein the data structure comprises a table, the table including an entry for each tier and bucket pairing.
4. The method of claim 3 , further comprising assigning each bucket and each tier a value in a hierarchical order, wherein territories in a bucket and a tier having a same value are considered to be properly placed.
5. The method of claim 4 , wherein territories in a bucket and a tier having a different value are considered to be candidates for moving.
6. The method of claim 1 , wherein partitioning the range of all possible information lifecycle management metrics comprises providing slack space within each tier to be used during data migrations.
7. A non-transitory computer storage medium having computer-executable instructions stored thereon which, when executed by a computer system, cause the computer system to:
subdivide a storage capacity of a storage system into a set of territories;
partition the set of territories into tiers;
partition a range of information lifecycle management metrics into buckets, each bucket corresponding to one of the tiers;
maintain a data structure relating the buckets and the tiers, the data structure containing an entry for each relationship of a bucket and a tier that is a number of territories currently stored in the tier having an information lifecycle management metric within the range of the bucket; and
move a territory from a current tier to a destination tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the destination tier.
8. The non-transitory computer storage medium of claim 7 , further causing the computer system to promote a territory from a lower tier to a higher tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the higher tier.
9. The non-transitory computer storage medium of claim 7 , further causing the computer system to demote a territory from a higher tier to a lower tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the lower tier.
10. The non-transitory computer storage medium of claim 7 , wherein the data structure comprises a table, the table including an entry for each tier and bucket pairing.
11. The non-transitory computer storage medium of claim 7 , further causing the computer system to determine a territory to move from a current tier to a destination tier using the data structure, wherein the territory to move is associated with an information lifecycle management metric outside the range of the bucket associated with the current tier.
12. The non-transitory computer storage medium of claim 7 , wherein partitioning the range of all possible information lifecycle management metrics comprises providing a periodic timer for updating the partitioning.
13. The non-transitory computer storage medium of claim 7 , wherein partitioning the range of all possible information lifecycle management metrics comprises establishing a partition based on a physical capacity of each tier, and a cumulative statistical distribution of territories with respect to information lifecycle management metrics.
14. A data storage system comprising:
a storage subsystem;
a plurality of physical storage devices associated with the storage subsystem; and
one or more data migration modules operable to cause the storage subsystem to:
subdivide a storage capacity of the storage system into a set of territories;
partition the set of territories into tiers;
partition a range of information lifecycle management metrics into buckets, each bucket corresponding to one of the tiers;
maintain a data structure relating the buckets and the tiers, the data structure containing an entry for each relationship of a bucket and a tier that is a number of territories currently stored in the tier having an information lifecycle management metric within the range of the bucket; and
move a territory from a current tier to a destination tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the destination tier,
wherein causing the storage system to move a territory comprises demoting a territory from a higher tier to a lower tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the lower tier.
15. The data storage system of claim 14 , wherein the data migration modules are further operable to cause the storage subsystem to promote a territory from a lower tier to a higher tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the higher tier.
16. The data storage system of claim 4 , wherein the data migration modules are further operable to cause the storage subsystem to demote a territory from a higher tier to a lower tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the lower tier.
17. The data storage system of claim 14 , wherein the data structure comprises a table, the table including an entry for each tier and bucket pairing.
18. A method for migrating data between tiers of a storage system, the method comprising:
subdividing a storage capacity of the storage system into a set of territories;
partitioning the set of territories into tiers;
partitioning a range of information lifecycle management metrics into buckets, each bucket corresponding to one of the tiers;
maintaining a data structure relating the buckets and the tiers, the data structure containing an entry for each relationship of a bucket and a tier that is a number of territories currently stored in the tier having an information lifecycle management metric within the range of the bucket;
moving a territory from a current tier to a destination tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the destination tier, and
determining a territory to move from a current tier to a destination tier using the data structure, wherein the territory to move is associated with an information lifecycle management metric outside the range of the bucket associated with the current tier.
19. Method for migrating data between tiers of a storage system, the method comprising:
subdividing a storage capacity of the storage system into a set of territories;
partitioning the set of territories into tiers;
partitioning a range of information lifecycle management metrics into buckets, each bucket corresponding to one of the tiers;
maintaining a data structure relating the buckets and the tiers, the data structure containing an entry for each relationship of a bucket and a tier that is a number of territories currently stored in the tier having an information lifecycle management metric within the range of the bucket; and
moving a territory from a current tier to a destination tier in response to the information lifecycle management metric associated with the territory being within the range of the bucket associated with the destination tier,
wherein partitioning the range of all possible information lifecycle management metrics comprises establishing a partition based on a physical capacity of each tier, and a cumulative statistical distribution of territories with respect to information lifecycle management metrics.