Storage system with efficient data and parity distribution across mixed-capacity storage devices
A storage system comprises mixed-capacity storage devices, including at least one or more storage devices having relatively low capacities and one or more storage devices having relatively high capacities. The storage system is configured to establish a redundant array of independent disks (RAID) arrangement comprising a plurality of stripes, with each of the plurality of stripes comprising a plurality of data blocks and one or more corresponding parity blocks, the data blocks and parity blocks being distributed across multiple ones of the storage devices. In conjunction with establishment of the RAID arrangement, the storage system is further configured to concentrate storage of the parity blocks in the one or more storage devices having the relatively high capacities. The storage devices in some embodiments comprise at least a first set of storage devices each having a first capacity and a second set of storage devices each having a second capacity higher than the first capacity.
1. An apparatus comprising:
a storage system comprising a plurality of storage devices;
the storage devices comprising at least one or more storage devices having relatively low capacities and one or more storage devices having relatively high capacities;
the storage system being configured:
to establish a redundant array of independent disks (RAID) arrangement comprising a plurality of stripes, with each of the plurality of stripes comprising a plurality of data blocks and one or more corresponding parity blocks, the data blocks and parity blocks being distributed across multiple ones of the storage devices; and
in conjunction with establishment of the RAID arrangement, to concentrate storage of the parity blocks in the one or more storage devices having the relatively high capacities;
wherein the storage devices comprise at least a first set of storage devices each having a first capacity and a second set of storage devices each having a second capacity higher than the first capacity, and wherein concentrating storage of the parity blocks in the one or more storage devices having the relatively high capacities comprises concentrating storage of the parity blocks in the second set of storage devices each having the second capacity higher than the first capacity; and
wherein concentrating storage of the parity blocks in the second set of storage devices comprises storing the parity blocks exclusively in the second set of storage devices, such that none of the parity blocks are stored in the first set of storage devices;
the first set of storage devices thereby storing no parity blocks of the RAID arrangement; and
the second set of storage devices thereby storing all of the parity blocks of the RAID arrangement.
2. The apparatus of claim 1 wherein the RAID arrangement comprises one of a RAID 5 arrangement supporting recovery from a failure of a single one of the plurality of storage devices, and a RAID 6 arrangement supporting recovery from simultaneous failure of up to two of the storage devices.
3. The apparatus of claim 1 wherein the same number of data blocks are stored in each of the storage devices of the first and second sets of storage devices.
4. The apparatus of claim 1 wherein an average read latency for reading data blocks of the storage devices is reduced by the concentrating relative to an average read latency that would otherwise result if the storage devices of the first set each had the same ratio of number of parity blocks to number of data blocks as the storage devices of the second set.
5. The apparatus of claim 1 wherein a rebuild time for reconstructing data blocks of at least one of the storage devices responsive to a failure of that storage device is reduced by the concentrating relative to a rebuild time that would otherwise result if the storage devices of the first set each had the same ratio of number of parity blocks to number of data blocks as the storage devices of the second set.
6. The apparatus of claim 1 wherein establishing the RAID arrangement comprises executing an algorithm to determine placement of the data blocks and the parity blocks across the multiple ones of the storage devices, the algorithm comprising:
a first phase to allocate data blocks across the multiple storage devices; and
a second phase to allocate parity blocks across at least a subset of the multiple storage devices using capacity remaining on those storage devices after allocation of data blocks to those storage devices.
7. An apparatus comprising:
a storage system comprising a plurality of storage devices;
the storage devices comprising at least one or more storage devices having relatively low capacities and one or more storage devices having relatively high capacities;
the storage system being configured:
to establish a redundant array of independent disks (RAID) arrangement comprising a plurality of stripes, with each of the plurality of stripes comprising a plurality of data blocks and one or more corresponding parity blocks, the data blocks and parity blocks being distributed across multiple ones of the storage devices; and
in conjunction with establishment of the RAID arrangement, to concentrate storage of the parity blocks in the one or more storage devices having the relatively high capacities;
wherein establishing the RAID arrangement comprises executing an algorithm to determine placement of the data blocks and the parity blocks across the multiple ones of the storage devices, the algorithm comprising:
a first phase to allocate data blocks across the multiple storage devices; and
a second phase to allocate parity blocks across at least a subset of the multiple storage devices using capacity remaining on those storage devices after allocation of data blocks to those storage devices; and
wherein the first phase of the algorithm comprises:
initializing an amount of allocated data for each storage device to zero;
initializing a total remaining data capacity to a collective data capacity of the plurality of storage devices, the collective data capacity comprising a total capacity of the storage devices less a desired parity capacity of the storage devices;
initializing a set of remaining storage devices to include all of the storage devices; and
for each storage device in the set of remaining storage devices, proceeding in an order from a storage device having a lowest capacity to a storage device having a highest capacity until all data blocks are allocated to storage devices:
responsive to the storage device having capacity sufficient to store an amount of data given by the total remaining data capacity divided by the number of storage devices in the set of remaining storage devices, allocating that amount of data to each of the storage devices in the set of remaining storage devices; and
otherwise allocating an amount of data given by remaining capacity of the storage device to that storage device and to all other storage devices in the set of remaining storage devices, subtracting the allocated amounts of data from the total remaining data capacity, and removing the storage device and any other storage devices with no remaining capacity from the set of remaining storage devices.
8. The apparatus of claim 6 wherein the second phase of the algorithm comprises:
determining remaining capacity for each of the storage devices; and
allocating parity blocks to the storage devices using their respective remaining capacities.
9. An apparatus comprising:
a storage system comprising a plurality of storage devices;
the storage devices comprising at least one or more storage devices having relatively low capacities and one or more storage devices having relatively high capacities;
the storage system being configured:
to establish a redundant array of independent disks (RAID) arrangement comprising a plurality of stripes, with each of the plurality of stripes comprising a plurality of data blocks and one or more corresponding parity blocks, the data blocks and parity blocks being distributed across multiple ones of the storage devices; and
in conjunction with establishment of the RAID arrangement, to concentrate storage of the parity blocks in the one or more storage devices having the relatively high capacities;
wherein establishing the RAID arrangement comprises executing an algorithm to determine placement of the data blocks and the parity blocks across the multiple ones of the storage devices, the algorithm comprising:
a first phase to allocate data blocks across the multiple storage devices; and
a second phase to allocate parity blocks across at least a subset of the multiple storage devices using capacity remaining on those storage devices after allocation of data blocks to those storage devices;
wherein the second phase of the algorithm comprises:
determining remaining capacity for each of the storage devices; and
allocating parity blocks to the storage devices using their respective remaining capacities; and
wherein allocating parity blocks to the storage devices comprises:
determining for each of at least a subset of the storage devices a corresponding current utilization of its available parity capacity; and
allocating parity blocks to particular ones of the storage devices based at least in part on the current utilizations of their respective available parity capacities;
wherein a storage device having a relatively low current utilization of its available parity capacity is prioritized over another storage device having a relatively high current utilization of its available parity capacity in the allocating of parity blocks to particular ones of the storage devices.
10. The apparatus of claim 1 wherein the storage system is further configured:
to detect a failure of at least one of the storage devices; and
responsive to the detected failure, to reconstruct data blocks of that storage device utilizing the data blocks and parity blocks stored on other ones of the storage devices.
11. A method for use in a storage system comprising a plurality of storage devices, the storage devices comprising at least one or more storage devices having relatively low capacities and one or more storage devices having relatively high capacities, the method comprising:
establishing a redundant array of independent disks (RAID) arrangement comprising a plurality of stripes, with each of the plurality of stripes comprising a plurality of data blocks and one or more corresponding parity blocks, the data blocks and parity blocks being distributed across multiple ones of the storage devices; and
in conjunction with establishment of the RAID arrangement, concentrating storage of the parity blocks in the one or more storage devices having the relatively high capacities;
wherein the storage devices comprise at least a first set of storage devices each having a first capacity and a second set of storage devices each having a second capacity higher than the first capacity, and wherein concentrating storage of the parity blocks in the one or more storage devices having the relatively high capacities comprises concentrating storage of the parity blocks in the second set of storage devices each having the second capacity higher than the first capacity; and
wherein concentrating storage of the parity blocks in the second set of storage devices comprises storing the parity blocks exclusively in the second set of storage devices, such that none of the parity blocks are stored in the first set of storage devices;
the first set of storage devices thereby storing no parity blocks of the RAID arrangement; and
the second set of storage devices thereby storing all of the parity blocks of the RAID arrangement.
12. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by a processor of a storage system comprising a plurality of storage devices, the storage devices comprising at least one or more storage devices having relatively low capacities and one or more storage devices having relatively high capacities, causes the storage system:
to establish a redundant array of independent disks (RAID) arrangement comprising a plurality of stripes, with each of the plurality of stripes comprising a plurality of data blocks and one or more corresponding parity blocks, the data blocks and parity blocks being distributed across multiple ones of the storage devices; and
in conjunction with establishment of the RAID arrangement, to concentrate storage of the parity blocks in the one or more storage devices having the relatively high capacities;
wherein the storage devices comprise at least a first set of storage devices each having a first capacity and a second set of storage devices each having a second capacity higher than the first capacity, and wherein concentrating storage of the parity blocks in the one or more storage devices having the relatively high capacities comprises concentrating storage of the parity blocks in the second set of storage devices each having the second capacity higher than the first capacity; and
wherein concentrating storage of the parity blocks in the second set of storage devices comprises storing the parity blocks exclusively in the second set of storage devices, such that none of the parity blocks are stored in the first set of storage devices;
the first set of storage devices thereby storing no parity blocks of the RAID arrangement; and
the second set of storage devices thereby storing all of the parity blocks of the RAID arrangement.
13. The computer program product of claim 12 wherein the same number of data blocks are stored in each of the storage devices of the first and second sets of storage devices.
14. The computer program product of claim 12 wherein an average read latency for reading data blocks of the storage devices is reduced by the concentrating relative to an average read latency that would otherwise result if the storage devices of the first set each had the same ratio of number of parity blocks to number of data blocks as the storage devices of the second set.
15. The computer program product of claim 12 wherein a rebuild time for reconstructing data blocks of at least one of the storage devices responsive to a failure of that storage device is reduced by the concentrating relative to a rebuild time that would otherwise result if the storage devices of the first set each had the same ratio of number of parity blocks to number of data blocks as the storage devices of the second set.
16. The computer program product of claim 12 wherein establishing the RAID arrangement comprises executing an algorithm to determine placement of the data blocks and the parity blocks across the multiple ones of the storage devices, the algorithm comprising:
a first phase to allocate data blocks across the multiple storage devices; and
a second phase to allocate parity blocks across at least a subset of the multiple storage devices using capacity remaining on those storage devices after allocation of data blocks to those storage devices.
17. The method of claim 11 wherein the same number of data blocks are stored in each of the storage devices of the first and second sets of storage devices.
18. The method of claim 11 wherein an average read latency for reading data blocks of the storage devices is reduced by the concentrating relative to an average read latency that would otherwise result if the storage devices of the first set each had the same ratio of number of parity blocks to number of data blocks as the storage devices of the second set.
19. The method of claim 11 wherein a rebuild time for reconstructing data blocks of at least one of the storage devices responsive to a failure of that storage device is reduced by the concentrating relative to a rebuild time that would otherwise result if the storage devices of the first set each had the same ratio of number of parity blocks to number of data blocks as the storage devices of the second set.
20. The method of claim 11 wherein establishing the RAID arrangement comprises executing an algorithm to determine placement of the data blocks and the parity blocks across the multiple ones of the storage devices, the algorithm comprising:
a first phase to allocate data blocks across the multiple storage devices; and
a second phase to allocate parity blocks across at least a subset of the multiple storage devices using capacity remaining on those storage devices after allocation of data blocks to those storage devices.