Configuring a data storage system by analyzing and grouping activity data of multiple LUNs
View Patent ↗Techniques for configuring a new data storage system generate IOPS distribution models from activity data of an existing data storage system and use the IOPS distribution models to generate a storage configuration for the new data storage system. Such techniques assign LUNs of the existing storage system to groups based on their input-out (I/O) characteristics and provide a storage configuration tailored to the expected IOPS distributions of each group of LUNs based on the IOPS distribution models and the activity data. The storage configurations also include RAID implementations tailored to each group of LUNs
1. A method of configuring a new data storage system, the method comprising:
receiving, by processing circuitry of a computing device, machine-readable activity data for a set of logical units of storage (LUNs) belonging to an existing data storage system, the activity data containing records of read and write operations directed to the set of LUNs, wherein each of the records of read and write operations directed to the set of LUNs includes an I/O size, the I/O size of each record indicating a number of bytes transferred as part of an I/O operation corresponding to that record;
assigning, by the processing circuitry, each LUN to one of multiple LUN groups based on the received activity data, each LUN group representing LUNs having similar input-output (I/O) characteristics to each other LUN in that group, wherein assigning each LUN to one of the multiple LUN groups based on the received activity data includes:
calculating, for at least one LUN, an average I/O size for a subset of the activity data corresponding to read and write operations directed to that LUN; and
assigning the at least one LUN to a LUN group based at least in part on the average I/O size of all the records for that LUN; and
outputting, by the processing circuitry, a storage configuration of the new data storage system by:
generating an IOPS (I/O operations per second) distribution model for each LUN group of the set of LUN groups using the activity data; and
including, as part of the storage configuration, a storage pool for each LUN group based on the IOPS distribution model for that LUN group and the activity data, the storage pool for each LUN group having a set of tiers supplying a set of respective tier capacities, each tier capacity supplied by a respective set of storage drives, each set of storage drives having a storage drive performance class, each storage drive performance class specifying a set of storage drive performance characteristics common to each storage drive having that storage drive performance class.
2. The method of claim 1 ,
wherein generating the IOPS distribution model for each LUN group includes:
ordering extents of storage capacity belonging to LUNs assigned to that LUN group according to a number of IOPS directed to each of the extents; and
providing a relationship between percentiles of extents of the LUNs assigned to that LUN group and cumulative percentages of IOPS directed to each percentile; and
wherein including, as part of the storage configuration, the storage pool for each LUN group based on the IOPS distribution model for that LUN group includes:
calculating, for that LUN group, a respective set of IOPS to be handled by each tier of the storage pool for that LUN group, based upon the IOPS distribution model for that LUN group and the activity data; and
converting the set of IOPS to be handled by each tier of the storage pool for that LUN group into the set of tier capacities for that LUN group according to the IOPS distribution model for that LUN group.
3. The method of claim 2 ,
wherein each of the records of read and write operations directed to the set of LUNs indicates whether a particular I/O request was fulfilled by accessing cache memory of the existing data storage system or directly accessing physical storage capacity of the existing data storage system; and
wherein assigning each LUN to one of the multiple LUN groups based on the received activity data further includes:
calculating, for at least one LUN, a cache-hit ratio for a subset of the activity data corresponding to read and write operations directed to that LUN, the cache hit ratio for that LUN indicating a percentage of I/O requests directed to that LUN fulfilled by accessing the cache memory; and
assigning the at least one LUN to a corresponding LUN group based at least in part on the calculated cache-hit ratio for that LUN.
4. The method of claim 3 , wherein assigning the at least one LUN to its corresponding LUN group based at least in part on the calculated cache-hit ratio for that LUN includes, in response to generating a comparison result indicating that the calculated cache-hit ratio is less than a predetermined threshold, outputting a LUN group to which that LUN will be assigned.
5. The method of claim 3 , wherein assigning the at least one LUN to its corresponding LUN group based at least in part on the calculated cache-hit ratio for that LUN includes, in response to generating a comparison result indicating that the calculated cache-hit ratio is greater than a predetermined threshold, outputting a LUN group to which that LUN will be assigned.
6. The method of claim 1 , wherein assigning the at least one LUN to its corresponding LUN group based at least in part on the calculated average I/O size for that LUN includes, in response to generating a comparison result indicating that the calculated average I/O size is less than a predetermined threshold, outputting a LUN group to which that LUN will be assigned.
7. The method of claim 1 , wherein assigning the at least one LUN to its corresponding LUN group based at least in part on the calculated average I/O size for that LUN includes, in response to generating a comparison result indicating that the calculated average I/O size is greater than a predetermined threshold, outputting a LUN group to which that LUN will be assigned.
8. The method of claim 1 , wherein including, as part of the storage configuration, the storage pool for each LUN group based on the IOPS distribution model for that LUN group includes providing a RAID configuration for that storage pool, the RAID configuration indicating how data will be arranged across disk extents of that storage pool and values for the tier capacities required for that RAID configuration.
9. The method of claim 1 ,
wherein the machine-readable activity data for the set of LUNs belonging to the existing data storage system includes a set of LUN descriptors, each LUN descriptor associated with a LUN of the set of LUNs belonging to the existing data storage system; and
wherein assigning each LUN to one of the multiple LUN groups based on the received activity data further includes assigning at least one LUN of the set of LUNs belonging to the existing data storage system to one of the multiple LUN groups based on the LUN descriptor associated with that LUN.
10. The method of claim 1 , wherein the method further comprises implementing the storage configuration by installing a set of physical storage drives in the new data storage system.
11. A computer program product comprising a set of non-transitory, computer-readable storage media storing instructions which, when executed by processing circuitry of a computing device, cause the processing circuitry to configure a new data storage system by:
receiving, by the processing circuitry, machine-readable activity data for a set of logical units of storage (LUNs) belonging to an existing data storage system, the activity data containing records of read and write operations directed to the set of LUNs, wherein each of the records of read and write operations performed by each LUN includes an I/O size, the I/O size of each record indicate a number of bytes transferred as part of an I/O operation corresponding to that record;
assigning, by the processing circuitry, each LUN to one of multiple LUN groups based on the received activity data, each LUN group representing LUNs having similar input-output (I/O) characteristics to each other LUN in that group, wherein assigning each LUN to one of multiple LUN groups based on the received activity data includes:
calculating, for at least one LUN, an average I/O size for a subset of the activity data corresponding to read and write operations directed to that LUN; and
assigning the at least one LUN to a corresponding LUN group based at least in part on the average I/O size of all the records for that LUN; and
outputting, by the processing circuitry, a storage configuration of the new data storage system by:
generating an IOPS distribution model for each LUN group of the set of LUN groups using the activity data; and
including, as part of the storage configuration, a storage pool for each LUN group based on the IOPS distribution model for that LUN group and the activity data, the storage pool for each LUN group having a set of tiers supplying a set of respective tier capacities, each tier capacity supplied by a respective set of storage drives, each set of storage drives having a storage drive performance class, each storage drive performance class specifying a set of storage drive performance characteristics common to each storage drive having that storage drive performance class.
12. The computer program product of claim 11 ,
wherein generating the IOPS distribution model for each LUN group includes:
ordering extents of storage capacity belonging to LUNs assigned to that LUN group according to a number of IOPS directed to each of the extents; and
providing a relationship between percentiles of extents of the LUNs assigned to that LUN group and cumulative percentages of IOPS directed to each percentile; and
wherein including, as part of the storage configuration, the storage pool for each LUN group based on the IOPS distribution model for that LUN group includes:
calculating, for that LUN group, a respective set of IOPS to be handled by each tier of the storage pool for that LUN group, based upon the IOPS distribution model for that LUN group and the activity data; and
converting the set of IOPS to be handled by each tier of the storage pool for that LUN group into the set of tier capacities for that LUN group according to the IOPS distribution model for that LUN group.
13. The computer program product of claim 12 ,
wherein each of the records of read and write operations performed by each LUN indicates whether a particular I/O request was fulfilled by accessing cache memory of the existing data storage system or directly accessing physical storage capacity of the existing data storage system; and
wherein assigning each LUN to one of the multiple LUN groups based on the received activity data further includes:
calculating, for at least one LUN, a cache-hit ratio for a subset of the activity data corresponding to read and write operations directed to that LUN, the cache hit ratio for that LUN indicating a percentage of I/O requests directed to that LUN fulfilled by accessing the cache memory; and
assigning the at least one LUN to a LUN group based at least in part on the calculated cache-hit ratio for that LUN.
14. The computer program product of claim 13 , wherein assigning the at least one LUN to its respective LUN group based at least in part on the calculated cache-hit ratio for that LUN includes:
generating a comparison result indicating whether the calculated cache-hit ratio of all the records for that LUN is less than a predetermined threshold ratio; and
selecting the corresponding LUN group for that LUN based on the comparison result.
15. The computer program product of claim 11 , wherein assigning the at least one LUN to its corresponding LUN group based at least in part on the average I/O size of all the records for that LUN includes:
generating a comparison result indicating whether the average I/O size of all the records for that LUN is less than a predetermined threshold size; and
selecting the corresponding LUN group for that LUN based on the comparison result.
16. The computer program product of claim 11 ,
wherein the machine-readable activity data for the set of LUNs belonging to the existing data storage system includes a set of LUN descriptors, each LUN descriptor associated with a LUN of the set of LUNs belonging to the existing data storage system; and
wherein assigning each LUN to one of multiple LUN groups based on the received activity data further includes assigning at least one LUN of the set of LUNs belonging to the existing data storage system to one of the multiple LUN groups based on the LUN descriptor associated with that LUN.
17. The computer program product of claim 11 , wherein including, as part of the storage configuration, the storage pool for each LUN group based on the IOPS distribution model for that LUN group includes providing a RAID configuration for that storage pool, the RAID configuration indicating how data will be arranged across disk extents of that storage pool and values for the tier capacities required for that RAID configuration.