Mapping storage extents into resiliency groups
Techniques for mapping large storage extents (“ubers”) into storage drive groups (“resiliency groups”) in a data storage system. The techniques can include, as the capacity of storage drives included in a first resiliency group is used up, forming a second resiliency group with one or more new storage drives. The disclosed techniques can further include allocating one or more storage drives from the first resiliency group to the second resiliency group to satisfy a requirement of a predetermined RAID storage configuration, and performing a reduced number of data movement operations to assure that data slices of one or more ubers initially assigned to the first resiliency group are allocated to storage drives in the same first or second resiliency group. In this way, data storage systems can be made to support a desired storage drive expansion with a reduced number of new storage drives.
1. A method of mapping ubers into resiliency groups in a data storage system, comprising:
as a capacity of storage drives allocated to a first resiliency group is used up, forming an additional second resiliency group with one or more new storage drives;
reallocating one or more of the storage drives from the first resiliency group to the second resiliency group to satisfy a requirement of a predetermined redundant array of independent disks (RAID) storage configuration, the reallocating of the storage drives causing data slices of one or more ubers to be distributed among storage drives in the first resiliency group and the second resiliency group;
performing a minimal number of data movement operations to move the data slices of each of the one or more ubers distributed among the storage drives in the first resiliency group and the second resiliency group to the same first or second resiliency group;
having reallocated the storage drives from the first resiliency group to the second resiliency group, determining, for each respective uber among the one or more ubers distributed in the first and second resiliency group, a first number of data slices of the respective uber that are allocated to storage drives in the first resiliency group and a second number of data slices of the respective uber that are allocated to storage drives in the second resiliency group; and
for each respective uber among the one or more ubers distributed in the first and second resiliency group, determining a maximum number among the first number and the second number of data slices of the respective uber.
2. The method of claim 1 further comprising:
having performed the minimal number of data movement operations to move the data slices of the respective ubers to the same first or second resiliency group, forming one or more sub-groups of storage drives in each of the first resiliency group and the second resiliency group, each sub-group of storage drives being arranged in the predetermined RAID storage configuration.
3. The method of claim 1 further comprising:
having determined the maximum number of the data slices of the respective uber, obtaining an available storage capacity of a respective resiliency group among the first resiliency group and the second resiliency group that corresponds to the maximum number of the data slices of the respective uber.
4. The method of claim 3 further comprising:
determining that the available storage capacity of the respective resiliency group is sufficient to store all of the data slices of the respective uber; and
assigning the respective uber to the respective resiliency group.
5. The method of claim 4 wherein the performing of the minimal number of data movement operations includes moving a minimum number of the data slices of the respective uber to the respective resiliency group to store all of the data slices of the respective uber in the respective resiliency group.
6. The method of claim 5 wherein the moving of the minimum number of the data slices of the respective uber to the respective resiliency group includes storing each of the data slices of the respective uber in a spare data slice of a respective storage drive in the respective resiliency group.
7. The method of claim 5 wherein the moving of the minimum number of the data slices of the respective uber to the respective resiliency group further includes determining that no spare data slices are available in the respective resiliency group, and storing each of the data slices of the respective uber in a reserved data slice of a respective storage drive in the respective resiliency group.
8. A data storage system, comprising:
a storage drive array including a plurality of storage drives allocated to a first resiliency group;
a memory; and
processing circuitry configured to execute program instructions out of the memory:
to form, as a capacity of storage drives allocated to a first resiliency group is used up, an additional second resiliency group with one or more new storage drives;
to reallocate one or more of the storage drives from the first resiliency group to the second resiliency group to satisfy a requirement of a predetermined redundant array of independent disks (RAID) storage configuration, thereby causing data slices of one or more ubers to be distributed among storage drives in the first resiliency group and the second resiliency group; and
to perform a minimal number of data movement operations to move the data slices of each of the one or more ubers distributed among the storage drives in the first resiliency group and the second resiliency group to the same first or second resiliency group;
having reallocated the storage drives from the first resiliency group to the second resiliency group, to determine, for each respective uber among the one or more ubers distributed in the first and second resiliency group, a first number of data slices of the respective uber that are allocated to storage drives in the first resiliency group and a second number of data slices of the respective uber that are allocated to storage drives in the second resiliency group; and
to determine, for each respective uber among the one or more ubers distributed in the first and second resiliency group, a maximum number among the first number and the second number of data slices of the respective uber.
9. The data storage system of claim 8 wherein the processing circuitry is further configured to execute the program instructions out of the memory:
having performed the minimal number of data movement operations to move the data slices of the respective ubers to the same first or second resiliency group, to form one or more sub-groups of storage drives in each of the first resiliency group and the second resiliency group, each sub-group of storage drives being arranged in the predetermined RAID storage configuration.
10. The data storage system of claim 8 wherein the processing circuitry is further configured to execute the program instructions out of the memory:
having determined the maximum number of the data slices of the respective uber, to obtain an available storage capacity of a respective resiliency group among the first resiliency group and the second resiliency group that corresponds to the maximum number of the data slices of the respective uber.
11. The data storage system of claim 10 wherein the processing circuitry is further configured to execute the program instructions out of the memory:
to determine that the available storage capacity of the respective resiliency group is sufficient to store all of the data slices of the respective uber; and
to assign the respective uber to the respective resiliency group.
12. The data storage system of claim 11 wherein the processing circuitry is further configured to execute the program instructions out of the memory:
to move a minimum number of the data slices of the respective uber to the respective resiliency group for storing all of the data slices of the respective uber in the respective resiliency group.
13. The data storage system of claim 12 wherein the processing circuitry is further configured to execute the program instructions out of the memory:
to store each of the data slices of the respective uber in a spare data slice of a respective storage drive in the respective resiliency group.
14. The data storage system of claim 12 wherein the processing circuitry is further configured to execute the program instructions out of the memory:
to determine that no spare data slices are available in the respective resiliency group; and
to store each of the data slices of the respective uber in a reserved data slice of a respective storage drive in the respective resiliency group.
15. The data storage system of claim 12 wherein the processing circuitry is further configured to execute the program instructions out of the memory:
to maintain, for each respective uber, indications of (i) an assignment of the respective uber to the respective resiliency group, and (ii) allocations of the data slices of the respective uber within the respective resiliency group, as data structure metadata in one or more of the memory and the storage drive array of the data storage system.
16. A computer program product including a set of non-transitory, computer-readable media having instructions that, when executed by processing circuitry of a data storage system, cause the processing circuitry to perform a method of mapping ubers into resiliency groups in the data storage system, the method comprising
as a capacity of storage drives allocated to a first resiliency group is used up, forming an additional second resiliency group with one or more new storage drives;
reallocating one or more of the storage drives from the first resiliency group to the second resiliency group to satisfy a requirement of a predetermined redundant array of independent disks (RAID) storage configuration, the reallocating of the storage drives causing data slices of one or more ubers to be distributed among storage drives in the first resiliency group and the second resiliency group;
performing a minimal number of data movement operations to move the data slices of each of the one or more ubers distributed among the storage drives in the first resiliency group and the second resiliency group to the same first or second resiliency group;
having reallocated the storage drives from the first resiliency group to the second resiliency group, determining, for each respective uber among the one or more ubers distributed in the first and second resiliency group, a first number of data slices of the respective uber that are allocated to storage drives in the first resiliency group and a second number of data slices of the respective uber that are allocated to storage drives in the second resiliency group; and
for each respective uber among the one or more ubers distributed in the first and second resiliency group, determining a maximum number among the first number and the second number of data slices of the respective uber.