Reorganizing disks and raid members to split a disk array during capacity expansion
A drive cluster with RAID (D+P) protection groups initially has D+P=W drives and W partitions. One protection group is created in each partition of the original drive cluster. The original drive cluster is scaled by selecting RAID protection group members starting with a first partition of the new drive and characterized by decrementing drive numbers and incrementing partition numbers, relocating the selected RAID protection group members within partitions such that a selected RAID protection group member on partition X of an existing drive is relocated to partition X of the new drive, and creating a new protection group in vacated partitions and the first partition of the new drive. The scaled drive cluster is split into independent smaller drive clusters when there are at least 2*W drives. RAID protection group members are relocated between a first smaller cluster that comprises sequentially numbered drives (W/2)+1 to (W/2)+W and a second smaller cluster that comprises the remaining drives until the second smaller cluster has the initial protection group distribution of the original drive cluster.
1. An apparatus, comprising:
at least one compute node comprising a processor and non-transitory memory;
drives that are managed by the at least one compute node; and
a drive cluster manager configured to:
sequentially number a first cluster of W of the drives,
organize each of the W drives into W partitions each having a fixed-size amount of storage capacity equal to storage capacity of other partitions of the first cluster,
implement redundant arrays of independent drives (RAID) protection groups on the drives, each RAID protection group having D data members and P parity members, where W is set equal to the sum of D and P, and the data members and parity members have an initial distribution to the partitions such that each partition of the first cluster contains members of a single RAID protection group,
scale the first cluster at least until there are 2*W drives, and split the scaled first cluster into a first smaller cluster that comprises sequentially numbered drives (W/2)+1 to (W/2)+W and a second smaller cluster that comprises remaining drives, and
adapt the second smaller cluster for scaling by adding more new drives.
2. The apparatus of claim 1 comprising the drive cluster manager being configured to select RAID protection group members starting with a first partition of the new drive and characterized by decrementing drive numbers and incrementing partition numbers, relocate the selected RAID protection group members within partitions such that a selected RAID protection group member on partition X of an existing drive is relocated to partition X of the new drive, and create a new protection group in vacated partitions and the first partition of the new drive, where X is a partition index.
3. The apparatus of claim 2 comprising the drive cluster manager being configured to relocate RAID protection group members between the first smaller cluster that comprises sequentially numbered drives (W/2)+1 to (W/2)+W and the second smaller cluster that comprises the remaining drives until the second smaller cluster has the initial distribution of the first cluster.
4. The apparatus of claim 3 comprising the drive cluster manager being configured to select a first set of RAID protection group members that are: members of the initial distribution of the first cluster; on the first smaller drive clusters that includes sequentially numbered drives (W/2)+1 to (W/2)+W; and can be directly relocated to free space on the second smaller drive cluster that comprises the remaining drives in accordance with the initial distribution of the first cluster.
5. The apparatus of claim 4 comprising the drive cluster manager being configured to relocate the selected first set of RAID group members to the second smaller drive cluster in accordance with the initial distribution of the first cluster.
6. The apparatus of claim 5 comprising the drive cluster manager being configured to select a second set of RAID protection group members that are: not members of the initial distribution of the first cluster and are on the second smaller drive cluster that comprises the remaining drives.
7. The apparatus of claim 6 comprising the drive cluster manager being configured to relocate the selected second set of RAID group members to the first smaller drive cluster.
8. The apparatus of claim 7 comprising the drive cluster manager being configured to select remaining RAID protection group members that are members of the first drive cluster and on the first smaller drive cluster and relocate the selected remaining protection group members to free space on the second smaller drive cluster in accordance with the initial distribution of the first cluster.
9. A method for scaling storage capacity of a storage array that comprises at least one compute node comprising a processor and non-transitory memory and drives that are managed by the at least one compute node, the method comprising:
sequentially numbering a first cluster of W of the drives;
organizing each of the W drives into W partitions each having a fixed-size amount of storage capacity equal to storage capacity of other partitions of the first cluster;
implementing redundant arrays of independent drives (RAID) protection groups on the drives, each RAID protection group having D data members and P parity members, where W is set equal to the sum of D and P, and the data members and parity members have an initial distribution to the partitions such that each partition of the first cluster contains members of a single RAID protection group;
scaling the first cluster at least until there are 2*W drives;
splitting the scaled first cluster into a first smaller cluster that comprises sequentially numbered drives (W/2)+1 to (W/2)+W and a second smaller cluster that comprises remaining drives; and
adapting the second smaller cluster for scaling by adding more new drives.
10. The method of claim 9 comprising selecting RAID protection group members starting with a first partition of the new drive and characterized by decrementing drive numbers and incrementing partition numbers, relocating the selected RAID protection group members within partitions such that a selected RAID protection group member on partition X of an existing drive is relocated to partition X of the new drive, and creating a new protection group in vacated partitions and the first partition of the new drive, where X is a partition index.
11. The method of claim 10 comprising relocating RAID protection group members between the first smaller cluster that comprises sequentially numbered drives (W/2)+1 to (W/2)+W and the second smaller cluster that comprises the remaining drives until the second smaller cluster has the initial distribution of the first cluster.
12. The method of claim 11 comprising selecting a first set of RAID protection group members that are: members of the initial distribution of the first cluster; on the first smaller drive clusters that includes sequentially numbered drives (W/2)+1 to (W/2)+W; and can be directly relocated to free space on the second smaller drive cluster that comprises the remaining drives in accordance with the initial distribution of the first cluster.
13. The method of claim 12 comprising relocating the selected first set of RAID group members to the second smaller drive cluster in accordance with the initial distribution of the first cluster.
14. The method of claim 13 comprising selecting a second set of RAID protection group members that are: not members of the initial distribution of the first cluster and are on the second smaller drive cluster that comprises the remaining drives.
15. The method of claim 14 comprising relocating the selected second set of RAID group members to the first smaller drive cluster.
16. The method of claim 15 comprising selecting remaining RAID protection group members that are members of the first drive cluster and on the first smaller drive cluster and relocating the selected remaining protection group members to free space on the second smaller drive cluster in accordance with the initial distribution of the first cluster.
17. A non-transitory computer-readable storage medium storing instructions that when executed by a compute node of a storage array cause the storage array to perform a method for scaling storage capacity of the storage array, the method comprising:
sequentially numbering a first cluster of W drives;
organizing each of the W drives into W partitions each having a fixed-size amount of storage capacity equal to storage capacity of other partitions of the first cluster;
implementing redundant arrays of independent drives (RAID) protection groups on the drives, each RAID protection group having D data members and P parity members, where W is set equal to the sum of D and P, and the data members and parity members have an initial distribution to the partitions such that each partition of the first cluster contains members of a single RAID protection group;
scaling the first cluster at least until there are 2*W drives;
splitting the scaled first cluster into a first smaller cluster that comprises sequentially numbered drives (W/2)+1 to (W/2)+W and a second smaller cluster that comprises remaining drives; and
adapting the second smaller cluster for scaling by adding more new drives.
18. The non-transitory computer-readable storage medium of claim 17 wherein the method further comprises relocating RAID protection group members between the first smaller cluster that comprises sequentially numbered drives (W/2)+1 to (W/2)+W and the second smaller cluster that comprises the remaining drives until the second smaller cluster has the initial distribution of the first cluster, where X is a partition index.
19. The non-transitory computer-readable storage medium of claim 18 wherein the method further comprises:
selecting a first set of RAID protection group members that are: members of the initial distribution of the first cluster; on the first smaller drive clusters that includes sequentially numbered drives (W/2)+1 to (W/2)+W; and can be directly relocated to free space on the second smaller drive cluster that comprises the remaining drives in accordance with the initial distribution of the first cluster; and
relocating the selected first set of RAID group members to the second smaller drive cluster in accordance with the initial distribution of the first cluster.
20. The non-transitory computer-readable storage medium of claim 19 wherein the method further comprises:
selecting a second set of RAID protection group members that are: not members of the initial distribution of the first cluster and are on the second smaller drive cluster that comprises the remaining drives;
relocating the selected second set of RAID group members to the first smaller drive cluster; and
selecting remaining RAID protection group members that are members of the first drive cluster and on the first smaller drive cluster and relocating the selected remaining protection group members to free space on the second smaller drive cluster in accordance with the initial distribution of the first cluster.