Storage system configured to guarantee sufficient capacity for a distributed raid rebuild process
View Patent ↗A storage system comprises a plurality of storage devices, and is configured to establish a redundant array of independent disks (RAID) arrangement comprising a plurality of stripes, with each of the plurality of stripes comprising a plurality of blocks, the blocks being distributed across multiple ones of the storage devices. In conjunction with establishment of the RAID arrangement, the storage system is further configured, for each of the plurality of stripes, to designate multiple ones of the storage devices as respective spare devices for that stripe, and for each of the storage devices, to determine numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices in each of multiple spare levels and for each of multiple failure combinations. A particular number of spare blocks is reserved for each of the storage devices using the determined numbers.
1. An apparatus comprising:
a storage system comprising a plurality of storage devices;
the storage system being configured:
to establish a redundant array of independent disks (RAID) arrangement comprising a plurality of stripes, with each of the plurality of stripes comprising a plurality of blocks, the blocks being distributed across multiple ones of the storage devices; and
in conjunction with establishment of the RAID arrangement:
for each of the plurality of stripes, to designate multiple ones of the storage devices as respective spare devices for that stripe;
for each of the storage devices, to determine numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices in each of multiple spare levels and for each of multiple failure combinations; and
to reserve a particular number of spare blocks for each of the storage devices based at least in part on the determined numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices across the multiple spare levels and the multiple failure combinations.
2. The apparatus of claim 1 wherein the RAID arrangement comprises a distributed RAID arrangement in which a total number of blocks per stripe is less than a total number of the storage devices over which the blocks of the plurality of stripes are distributed.
3. The apparatus of claim 1 wherein the RAID arrangement comprises at least one parity RAID arrangement supporting recovery from a failure of at least one of the plurality of storage devices and wherein a given one of the stripes comprises a plurality of data blocks and one or more parity blocks.
4. The apparatus of claim 1 wherein the storage system is further configured to store stripe metadata indicating for each of the stripes the multiple ones of the storage devices that are designated as respective spare devices for that stripe.
5. The apparatus of claim 4 wherein the stripe metadata indicating for each of the stripes the multiple ones of the storage devices that are designated as respective spare devices for that stripe comprises an allocation table, the allocation table comprising a plurality of entries for respective ones of the stripes with each such entry comprising identifiers of the storage devices that are designated as respective spare devices for that stripe.
6. The apparatus of claim 1 wherein the storage system is further configured to store stripe metadata indicating for each of the storage devices the determined numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices in each of multiple spare levels and for each of multiple failure combinations.
7. The apparatus of claim 6 wherein the stripe metadata indicating for each of the storage devices the determined numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices in each of multiple spare levels and for each of multiple failure combinations comprises a reserve table, the reserve table comprising a plurality of entries for respective ones of the storage devices with each such entry comprising the determined numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices in each of multiple spare levels and for each of multiple failure combinations.
8. The apparatus of claim 1 wherein reserving a particular number of spare blocks for each of the storage devices based at least in part on the determined numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices across the multiple spare levels and the multiple failure combinations comprises:
summing the determined numbers for the storage device across the spare levels to generate resulting sums for respective ones of the failure conditions;
determining a highest one of the resulting sums across the respective ones of the failure conditions; and
reserving a particular number of spare blocks for the storage device that is greater than or equal to the highest one of the resulting sums.
9. The apparatus of claim 1 wherein designating multiple ones of the storage devices as respective spare devices for a given one of the stripes comprises designating at least first and second storage devices as respective first and second spare devices for the given stripe, the first and second storage devices corresponding to respective first and second spare levels of the multiple spare levels, and wherein a first failure impacting the given stripe results in utilization of a spare block of the first spare device and a second failure impacting the given stripe results in utilization of a spare block of the second spare device.
10. The apparatus of claim 1 wherein the failure combinations comprise respective different sets of two or more of the storage devices.
11. The apparatus of claim 1 wherein different ones of the storage devices are implemented on different ones of a plurality of nodes of a distributed storage system.
12. The apparatus of claim 11 wherein the failure combinations comprise different combinations of one of the storage devices and one of the nodes.
13. The apparatus of claim 1 wherein the storage system is further configured:
to detect a failure of at least one of the storage devices; and
responsive to the detected failure, to initiate a rebuild process to reconstruct blocks of the one or more failed storage devices utilizing the blocks of other ones of the storage devices;
wherein the rebuild process utilizes at least a subset of the reserved spare blocks of respective ones of the non-failed storage devices.
14. The apparatus of claim 1 wherein the reserved spare blocks are sufficient in number to guarantee that a rebuild process initiated responsive to a failure of one or more of the storage devices has sufficient available storage device capacity to reconstruct the blocks of the one or more failed storage devices.
15. A method for use in a storage system comprising a plurality of storage devices, the method comprising:
establishing a redundant array of independent disks (RAID) arrangement comprising a plurality of stripes, with each of the plurality of stripes comprising a plurality of blocks, the blocks being distributed across multiple ones of the storage devices; and
in conjunction with establishment of the RAID arrangement:
for each of the plurality of stripes, designating multiple ones of the storage devices as respective spare devices for that stripe;
for each of the storage devices, determining numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices in each of multiple spare levels and for each of multiple failure combinations; and
reserving a particular number of spare blocks for each of the storage devices based at least in part on the determined numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices across the multiple spare levels and the multiple failure combinations.
16. The method of claim 15 wherein reserving a particular number of spare blocks for each of the storage devices based at least in part on the determined numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices across the multiple spare levels and the multiple failure combinations comprises:
summing the determined numbers for the storage device across the spare levels to generate resulting sums for respective ones of the failure conditions;
determining a highest one of the resulting sums across the respective ones of the failure conditions; and
reserving a particular number of spare blocks for the storage device that is greater than or equal to the highest one of the resulting sums.
17. The method of claim 15 wherein designating multiple ones of the storage devices as respective spare devices for a given one of the stripes comprises designating at least first and second storage devices as respective first and second spare devices for the given stripe, the first and second storage devices corresponding to respective first and second spare levels of the multiple spare levels, and wherein a first failure impacting the given stripe results in utilization of a spare block of the first spare device and a second failure impacting the given stripe results in utilization of a spare block of the second spare device.
18. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by a processor of a storage system comprising a plurality of storage devices, causes the storage system:
to establish a redundant array of independent disks (RAID) arrangement comprising a plurality of stripes, with each of the plurality of stripes comprising a plurality of blocks, the blocks being distributed across multiple ones of the storage devices; and
in conjunction with establishment of the RAID arrangement:
for each of the plurality of stripes, to designate multiple ones of the storage devices as respective spare devices for that stripe;
for each of the storage devices, to determine numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices in each of multiple spare levels and for each of multiple failure combinations; and
to reserve a particular number of spare blocks for each of the storage devices based at least in part on the determined numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices across the multiple spare levels and the multiple failure combinations.
19. The computer program product of claim 18 wherein reserving a particular number of spare blocks for each of the storage devices based at least in part on the determined numbers of the stripes for which that storage device is designated as a spare device for respective ones of the other storage devices across the multiple spare levels and the multiple failure combinations comprises:
summing the determined numbers for the storage device across the spare levels to generate resulting sums for respective ones of the failure conditions;
determining a highest one of the resulting sums across the respective ones of the failure conditions; and
reserving a particular number of spare blocks for the storage device that is greater than or equal to the highest one of the resulting sums.
20. The computer program product of claim 18 wherein designating multiple ones of the storage devices as respective spare devices for a given one of the stripes comprises designating at least first and second storage devices as respective first and second spare devices for the given stripe, the first and second storage devices corresponding to respective first and second spare levels of the multiple spare levels, and wherein a first failure impacting the given stripe results in utilization of a spare block of the first spare device and a second failure impacting the given stripe results in utilization of a spare block of the second spare device.