Scaling raid-based storage by redistributing splits
Each drive in a cluster is organized into splits of a single size. The splits are used as protection group members for a selected RAID level. Individual clusters can be scaled up with a number of new drives that is less than the number of protection group members for the RAID level by redistributing some data/parity to splits of the new drives. Splits are selected for redistribution such that new protection groups are created. If the number of new drives is equal to or greater than the number of protection group members for the RAID level, then new clusters may be created using the new drives. Any remaining drives are used to scale-up individual clusters.
1. An apparatus, comprising:
at least one computing node comprising a processor and non-transitory memory;
clusters of drives that are managed by the at least one computing node; and
computer program code on the non-transitory memory, executed by the processor, comprising:
instructions that organize each of the drives into a plurality of splits, wherein each split is a fixed-size amount of storage capacity, and all splits have equal storage capacity;
instructions that implement a selected RAID (Redundant Array of Independent Drives) level using a first number of the splits as RAID protection group members, where the first number indicates how many members are in each protection group of the RAID level;
instructions that scale RAID storage capacity in response to addition of a second number of new drives, comprising:
instructions that create one or more new clusters, each of the new clusters comprising the first number of the new drives, leaving a third number of remaining new drives, where the third number is less than the first number;
instructions that add the remaining new drives to a selected cluster, comprising:
instructions that select splits of drives of the selected cluster and redistribute the selected splits to the remaining new drives added to the selected cluster; and
instructions that create at least one new protection group in the selected cluster using at least some splits freed by redistribution of the selected splits to the remaining new drives added to the selected cluster.
2. The apparatus of claim 1 wherein the instructions that scale RAID storage capacity create a maximum possible number of new clusters from the second number of new drives.
3. The apparatus of claim 2 wherein the instructions that add the remaining new drives to the selected cluster add only drives remaining from drives allocated for creation of the maximum possible number of new clusters from the second number of new drives.
4. The apparatus of claim 1 wherein the instructions that add the remaining the new drives to the selected cluster operate to select the cluster to minimizes differences in numbers of drives between clusters.
5. The apparatus of claim 1 wherein the instructions that add fewer than the first number of the new drives to the selected cluster add a single new drive to the selected cluster.
6. The apparatus of claim 1 wherein the selected cluster initially includes conceptual matrices with an upper-left triangle and a lower-right triangle, and after m new drives are added to the selected cluster and splits from the lower-right triangle are redistributed to the m new drives, space is created for m new protection groups in a diagonal conceptual matrix.
7. A method, comprising:
organizing individual storage drives into a plurality of splits, wherein each split is a fixed-size amount of storage capacity and all splits have equal storage capacity;
implementing a selected RAID (Redundant Array of Independent Drives) level using a first number of the splits as RAID protection group members, where the first number indicates how many members are in each protection group of the RAID level;
scaling RAID storage capacity in response to addition of a second number of new drives, comprising:
creating one or more new clusters, each of the new clusters comprising the first number of the new drives, leaving a third number of remaining new drives, where the third number is less than the first number;
adding the remaining new drives to a selected cluster of drives;
selecting splits of drives of the selected cluster and redistributing the selected splits to the remaining new drives added to the selected cluster; and
creating at least one new protection group in the selected cluster using at least some splits freed by redistribution of the selected splits to the remaining new drives added to the selected cluster.
8. The method of claim 7 comprising creating a maximum possible number of new clusters from the second number of new drives.
9. The method of claim 8 comprising adding to the selected cluster only drives remaining from the second number of new drives minus the new drives allocated for creating the maximum possible number of new clusters from the second number of new drives.
10. The method of claim 7 comprising selecting the cluster to minimizes differences in numbers of drives between clusters.
11. The method of claim 7 comprising adding a single new drive to the selected cluster.
12. The apparatus of claim 7 wherein the selected cluster initially includes conceptual matrices with an upper-left triangle and a lower-right triangle and, after m new drives are added to the selected cluster, redistributing splits from the lower-right triangle to the m new drives to create space for m new protection groups in a diagonal conceptual matrix.
13. A scalable storage array, comprising:
clusters of drives that are each organized into a plurality of splits, wherein each split is a fixed-size amount of storage capacity and all splits have equal storage capacity, a selected RAID (Redundant Array of Independent Drives) level using a first number of the splits as RAID protection group members, where the first number indicates how many members are in each protection group of the RAID level; and
at least one computing node that scales RAID storage capacity in response to addition of a second number of new drives by creating one or more new clusters, each of the new clusters comprising the first number of the new drives, leaving a third number of remaining new drives, where the third number is less than the first number; adding the remaining new drives to a selected cluster of drives; selecting splits of drives of the selected cluster and redistributing the selected splits to the remaining new drives added to the selected cluster, and creating at least one new protection group in the selected cluster using at least some splits freed by redistribution of the selected splits to the remaining the new drives added to the selected cluster.
14. The scalable storage array of claim 13 wherein the at least one computing node creates a maximum possible number of new clusters from the second number of new drives.
15. The scalable storage array of claim 14 wherein the at least one computing node adds only drives remaining from drives allocated for creation of the maximum possible number of new clusters from the second number of new drives.
16. The scalable storage array of claim 13 wherein the at least one computing node selects the cluster to minimizes differences in numbers of drives between clusters.
17. The scalable storage array of claim 13 wherein the at least one computing node adds a single new drive to the selected cluster.