Raid reliability with a provisional spare disk
Disks of equal storage capacity in a disk cluster have M*W partitions, where RAID width W=D+P. RAID (D+P) protection groups are implemented on the disks with protection group members stored in individual partitions. An amount of storage capacity equal to the storage capacity of one disk is distributed across multiple disks in spare partitions. The spare partitions may be distributed such that no more than one spare partition resides on a single disk. M may be selected to optimize rebuild latency. The protection group members of a failed disk are rebuilt in the distributed spare partitions and subsequently relocated to a provisional spare drive. The populated provisional spare drive replaces the failed drive.
1. An apparatus, comprising:
a storage array comprising:
at least one compute node comprising at least one processor and non-transitory computer-readable memory;
a plurality of data storage disks that are accessed by the at least one compute node and used to store data that is accessed via the at least one compute node, each of the data storage disks being associated with one disk cluster that contains Redundant Array of Independent Disks (RAID) (D+P) protection groups where RAID width W=D+P and each disk has M*W indexed partitions of equal storage capacity and each member of ones of the RAID protection groups is located on one of the partitions on a different one of the plurality of data storage disks;
a provisional spare disk that is present in the storage array prior to failure of one of the data storage disks of the disk cluster; and
a disk manager configured to:
distribute spare partitions across a plurality of the disk clusters; and
responsive to detection of a failed one of the data storage disks:
rebuild protection group members of the failed data storage disk on the distributed spare partitions such that each rebuilt member of ones of the RAID protection groups of the failed data storage disk is located on one of the partitions on a different one of the plurality of data storage disks; and
relocate rebuilt protection group members from the distributed spare partitions to the provisional spare disk.
2. The apparatus of claim 1 wherein the disk manager is configured to replace the failed data storage disk with the provisional spare disk having the rebuilt protection group members stored thereon.
3. The apparatus of claim 1 wherein the disk manager is configured to select a value of M to optimize latency of rebuild of protection group members of the failed data storage disk on the distributed spare partitions.
4. The apparatus of claim 1 wherein the disk manager is configured to distribute spare partitions across the plurality of the disk clusters until none of the data storage disks includes more than one distributed spare partition.
5. The apparatus of claim 4 wherein the disk manager is configured to reclaim partitions freed as a result of distribution of spare partitions.
6. The apparatus of claim 5 wherein the disk manager is configured to create a new RAID (D+P) protection group using the reclaimed partitions.
7. The apparatus of claim 1 wherein the disk manager is configured to rebuild protection group members of a second failed data storage disk on the distributed spare partitions after the rebuilt protection group members are relocated from the distributed spare partitions to the provisional spare disk.
8. A method implemented by a storage array comprising at least one compute node comprising at least one processor and non-transitory computer-readable memory and a plurality of data storage disks that are accessed by the at least one compute node and used to store data that is accessed via the at least one compute node, each of the data storage disks being associated with one disk cluster that contains Redundant Array of Independent Disks (RAID) (D+P) protection groups where RAID width W=D+P and each disk has M*W indexed partitions of equal storage capacity and each member of ones of the RAID protection groups is located on one of the partitions on a different one of the plurality of data storage disks, the method comprising:
distributing spare partitions across a plurality of the disk clusters; and
responsive to detection of a failed one of the data storage disks:
rebuilding protection group members of the failed data storage disk on the distributed spare partitions; and
relocating rebuilt protection group members from the distributed spare partitions to the provisional spare disk, wherein the provisional spare disk is present in the storage array prior to detection of the failed one of the data storage disks.
9. The method of claim 8 comprising replacing the failed data storage disk with the provisional spare disk having the rebuilt protection group members stored thereon.
10. The method of claim 8 comprising selecting a value of M to optimize latency of rebuild of protection group members of the failed data storage disk on the distributed spare partitions.
11. The method of claim 8 comprising distributing spare partitions across the plurality of the disk clusters until none of the data storage disks includes more than Off one distributed spare partition.
12. The method of claim 11 comprising reclaiming partitions freed as a result of distribution of spare partitions.
13. The method of claim 12 comprising creating a new RAID (D+P) protection group using the reclaimed partitions.
14. The method of claim 8 comprising rebuilding protection group members of a second failed data storage disk on the distributed spare partitions after the rebuilt protection group members are relocated from the distributed spare partitions to the provisional spare disk.
15. A non-transitory computer-readable storage medium storing instructions that when executed by a storage array comprising at least one compute node comprising at least one processor and non-transitory computer-readable memory and a plurality of data storage disks that are accessed by the at least one compute node and used to store data that is accessed via the at least one compute node, each of the data storage disks being associated with one disk cluster that contains Redundant Array of Independent Disks (RAID) RAID (D+P) protection groups where RAID width W=D+P and each disk has M*W indexed partitions of equal storage capacity and each member of ones of the RAID protection groups is located on one of the partitions on a different one of the plurality of data storage disks, cause the storage array to perform a method for recovering from disk failure, the method comprising:
distributing spare partitions across a plurality of the disk clusters; and
responsive to detection of a failed one of the data storage disks:
rebuilding protection group members of the failed data storage disk on the distributed spare partitions; and
relocating rebuilt protection group members from the distributed spare partitions to the provisional spare disk, wherein the provisional spare disk is present in the storage array prior to detection of the failed one of the data storage disks.
16. The non-transitory computer-readable storage medium of claim 15 wherein the method further comprises replacing the failed data storage disk with the provisional spare disk having the rebuilt protection group members stored thereon.
17. The non-transitory computer-readable storage medium of claim 15 wherein the method further comprises selecting a value of M to optimize latency of rebuild of protection group members of the failed data storage disk on the distributed spare partitions.
18. The non-transitory computer-readable storage medium of claim 15 wherein the method further comprises distributing spare partitions across the plurality of the disk clusters until none of the data storage disks includes more than one distributed spare partition.
19. The non-transitory computer-readable storage medium of claim 18 wherein the method further comprises reclaiming partitions freed as a result of distribution of spare partitions.
20. The non-transitory computer-readable storage medium of claim 19 wherein the method further comprises creating a new RAID (D+P) protection group using the reclaimed partitions.