Predictive redistribution of capacity in a flexible RAID system
A storage array includes a scalable drive cluster and non-scaling drive clusters on which RAID (D+P) protection groups are implemented using partition as protection group members. The scalable drive cluster is scaled by adding one or more new drives and moving protection group members onto the new drives. Reserve capacity is calculated based on scaling of the scalable drive cluster by converting unused partitions to reserve capacity when the cluster is scaled. When W=(D+P) new drives are added to the storage array the reserve capacity is replenished, if any was used, by moving protection groups from the scalable drive cluster to a non-scaling drive cluster created with the W new drives. Maintaining the reserve capacity on the scaling drive cluster improves function of the storage array because unutilized reserve capacity can be relocated during scaling without moving data.
1. An apparatus, comprising:
a storage array comprising:
at least one compute node comprising at least one processor and non-transitory memory;
a plurality of data storage drives that are accessed by the at least one compute node and used to store data that is accessed via the at least one compute node; and
a drive manager configured to:
create a scalable drive cluster from W of the drives each having W partitions each having a fixed-size amount of storage capacity equal to storage capacity of other partitions of the scalable drive cluster, wherein the drive manager implements redundant arrays of independent drives (RAID) each having D+P=W RAID protection group members that are initially symmetrically distributed to the partitions and no more than one member of a protection group is located on a single one of the drives;
create reserve capacity using at least some of the partitions of the scalable drive cluster;
scale the scalable drive cluster by adding at least one new drive to the scalable drive cluster and relocating at least some of the protection group members and reserve capacity to the at least one new drive; and
replenish a utilized portion of the reserve capacity;
whereby functioning of the storage array is improved by reducing data movement associated with scaling of the scalable drive cluster.
2. The apparatus of claim 1 wherein the drive manager calculates reserve capacity in response to addition of at least one new drive to the W drives of the scalable drive cluster.
3. The apparatus of claim 2 wherein the drive manager creates the reserve capacity via conversion of unused capacity on the W drives of the scalable drive cluster to reserve capacity in response to addition of at least one new drive to the W drives of the scalable drive cluster.
4. The apparatus of claim 1 wherein the drive manager is configured to replenish the utilized portion of the reserve capacity by moving at least one protection group to new drives that are not members of the scalable drive cluster.
5. The apparatus of claim 4 wherein the new drives are part of a group of at least W new drives that are used to create a non-scaling drive cluster.
6. The apparatus of claim 1 wherein for each new drive that is added to the scalable cluster in ordered succession a lowest numbered unrotated partition of the W drives is rotated onto the new drive such that a protection group member at drive Dn is moved to partition Pn of the new drive.
7. The apparatus of claim 1 wherein the drive manager symmetrically distributes protection groups by assigning a member of a protection group m to a drive partition at drive X, partition Y using the algorithm: m=(X+Y) MODULO (D+P).
8. A method implemented by a storage array with at least one compute node and a plurality of data storage drives that are accessed by the at least one compute node and used to store data that is accessed via the at least one compute node, the method comprising:
creating a scalable drive cluster from W of the drives, each of the drives having W partitions, each of the partitions having a fixed-size amount of storage capacity equal to storage capacity of other partitions of the scalable drive cluster;
implementing redundant arrays of independent drives (RAID) each having D+P=W RAID protection group members that are initially symmetrically distributed to the partitions with no more than one member of a protection group located on a single one of the drives;
creating reserve capacity using at least some of the partitions of the scalable drive cluster;
scaling the scalable drive cluster by adding at least one new drive to the scalable drive cluster and relocating at least some of the protection group members and reserve capacity to the at least one new drive; and
replenishing a utilized portion of the reserve capacity;
whereby functioning of the storage array is improved by reducing data movement associated with scaling of the scalable drive cluster.
9. The method of claim 8 comprising calculating reserve capacity in response to addition of at least one new drive to the W drives of the scalable drive cluster.
10. The method of claim 9 comprising creating the reserve capacity by converting unused capacity on the W drives of the scalable drive cluster to reserve capacity in response to addition of at least one new drive to the W drives of the scalable drive cluster.
11. The method of claim 8 comprising replenishing the utilized portion of the reserve capacity by moving at least one protection group to new drives that are not members of the scalable drive cluster.
12. The method of claim 11 comprising utilizing the new drives to create a non-scaling drive cluster.
13. The method of claim 8 comprising, for each new drive that is added to the scalable cluster in ordered succession, rotating a lowest numbered unrotated partition of the W drives onto the new drive such that a protection group member at drive Dn is moved to partition Pn of the new drive.
14. The method of claim 8 comprising symmetrically distributing protection groups by assigning each member of a protection group m to a drive partition at drive X, partition Y using the algorithm: m=(X+Y) MODULO (D+P).
15. A non-transitory computer-readable storage medium storing instructions that when executed by a compute node cause the compute node to perform a method for using a storage array to reduce data movement associated with scaling of a scalable drive cluster, the method comprising:
creating a scalable drive cluster from W of the drives, each of the drives having W partitions, each of the partitions having a fixed-size amount of storage capacity equal to storage capacity of other partitions of the scalable drive cluster;
implementing redundant arrays of independent drives (RAID) each having D+P=W RAID protection group members that are initially symmetrically distributed to the partitions with no more than one member of a protection group located on a single one of the drives;
creating reserve capacity using at least some of the partitions of the scalable drive cluster; scaling the scalable drive cluster by adding at least one new drive to the scalable drive cluster and relocating at least some of the protection group members and reserve capacity to the at least one new drive; and
replenishing a utilized portion of the reserve capacity;
whereby functioning of the storage array is improved by reducing data movement associated with scaling of the scalable drive cluster.
16. The non-transitory computer-readable storage medium of claim 15 wherein the method further comprises calculating reserve capacity in response to addition of at least one new drive to the W drives of the scalable drive cluster.
17. The non-transitory computer-readable storage medium of claim 16 wherein the method further comprises creating the reserve capacity by converting unused capacity on the W drives of the scalable drive cluster to reserve capacity in response to addition of at least one new drive to the W drives of the scalable drive cluster.
18. The non-transitory computer-readable storage medium of claim 15 wherein the method further comprises replenishing the utilized portion of the reserve capacity by moving at least one protection group to new drives that are not members of the scalable drive cluster.
19. The non-transitory computer-readable storage medium of claim 18 wherein the method further comprises utilizing the new drives to create a non-scaling drive cluster.
20. The non-transitory computer-readable storage medium of claim 15 wherein the method further comprises, for each new drive that is added to the scalable cluster in ordered succession, rotating a lowest numbered unrotated partition of the W drives onto the new drive such that a protection group member at drive Dn is moved to partition Pn of the new drive.