Storage system configuration based on workload characteristics and performance metrics
One or more aspects of the present disclosure relate to providing storage system configuration recommendations. System configurations of one or more storage devices can be determined based on their respective collected telemetry information. Performance of storage devices having different system configurations can be predicted based on one or more of: the collected telemetry information and each of the different system configurations. In response to receiving one or more requested performance characteristics and workload conditions, one or more recommended storage device configurations can be provided for each request based on the predicted performance characteristics, the requested performance characteristics, and the workload conditions.
1. An apparatus comprising at least one processor configured to:
determine system configurations of each of one or more storage devices based on their respective collected telemetry information;
predict performance characteristics of each of a plurality of storage devices having different system configurations based on one or more of: the collected telemetry information and each of the different system configurations, wherein the apparatus is further configured to:
identify input/output (I/O) data types related to an I/O workload received by a storage array,
generate response time (RT) data pairs, mapping each uniquely available system configuration and one or more of the I/O workload and I/O data types to respective response times, and
in response to receiving one or more requested performance characteristics and current workload conditions, provide one or more recommended storage device configurations for each request based on the predicted performance characteristics, the requested performance characteristics, and the workload conditions.
2. The apparatus of claim 1 , wherein the telemetry information is collected from at least one or more of: one or more field-deployed storage devices or one or more lab operated storage device.
3. The apparatus of claim 1 , wherein collecting the telemetry information includes controlling a telemetry collection device in each storage device to collect and transmit the telemetry information during one or more predetermined time-windows.
4. The apparatus of claim 3 , wherein:
the telemetry information includes one or more of: each storage device's system configuration, each storage device's input/output (I/O) workloads, and each storage device's performance characteristics associated with the workload conditions; and
wherein each storage device's performance characteristics are defined at least by their respective response times.
5. The apparatus of claim 4 further configured to:
segment the telemetry information based on one or more of: industry, workload types, performance characteristics, input/output (I/O) operations, and service level (SL) performance tiers of each storage device;
wherein each storage device's response times are segmented into each storage device's SL performance tiers; and
wherein workload conditions are characterized at least by amounts of read and write (R/W) I/O operations of each storage device's workload.
6. The apparatus of claim 4 further configured to:
for one or more sets of storage devices, predict performance characteristics by at least using one or more prediction models built for each storage device, wherein each prediction model is built by processing each storage device's telemetry information using one or more machine learning techniques; and
wherein:
the one or more machine learning techniques includes at least a non-linear regression learning technique, amongst other machine learning techniques, and
the non-linear regression technique includes at least one or more of a random forest and a neural network, amongst other machine learning techniques.
7. The apparatus of claim 6 further configured to:
predict the performance characteristics based on at least one or more of each storage device's system configurations and workloads present in the collected telemetry information, amongst other data;
process the telemetry information using the one or more machine learning techniques augmented with each storage device's uncollected telemetry information and validated telemetry information.
8. The apparatus of claim 7 , wherein each storage device's uncollected and validated telemetry information includes at least one or more of each storage device's validated system configurations and validated workloads not present in each storage device's collected telemetry information.
9. The apparatus of claim 1 further configured to:
for one or more sets of storage device, build one or more system configuration prediction models to determine probabilities of each recommended storage device configuration meeting each storage device's SL performance tiers under one or more different workload conditions.
10. The apparatus of claim 9 , wherein:
each different workload condition is based on one or more anticipated workloads; and
each of the one or more anticipated workloads is predicted using one or more machine learning techniques to process one or more of: a customer type associated with each requested performance characteristic, the collected telemetry information, and uncollected telemetry information.
11. A method comprising:
determining system configurations of each of one or more storage devices based on their respective collected telemetry information;
predicting performance characteristics of each of a plurality of storage devices having different system configurations based on one or more of: the collected telemetry information and each of the different system configurations, wherein predicting performance characteristics further includes:
identifying input/output (I/O) data types related to an I/O workload received by a storage array,
generating response time (RT) data pairs, mapping each uniquely available system configuration and one or more of the I/O workload and I/O data types to respective response times, and
in response to receiving one or more requested performance characteristics and current workload conditions, providing one or more recommended storage device configurations for each request based on the predicted performance characteristics, the requested performance characteristics, and the workload conditions.
12. The method of claim 11 , wherein the telemetry information is collected from at least one or more of: one or more field-deployed storage devices or one or more lab operated storage devices.
13. The method of claim 11 , wherein collecting the telemetry information includes controlling a telemetry collection device in each storage device to collect and transmit the telemetry information during one or more predetermined time-windows.
14. The method of claim 13 , wherein:
the telemetry information includes one or more of: each storage device's system configuration, each storage device's input/output (I/O) workloads, and each storage device's performance characteristics associated with the workload conditions; and
wherein each storage device's performance characteristics are defined at least by their respective response times.
15. The method of claim 14 further comprising:
segmenting the telemetry information based on one or more of: industry, workload types, performance characteristics, input/output (I/O) operations, and service level (SL) performance tiers of each storage device;
wherein each storage device's response times are segmented into each storage device's SL performance tiers; and
wherein workload conditions are characterized at least by amounts of read and write (R/W) I/O operations of each storage device's workload.
16. The method of claim 14 , wherein:
predicting each storage device's performance characteristics includes using one or more prediction models built for each storage device, wherein each prediction model is built by processing each storage device's telemetry information using one or more machine learning techniques;
the one or more machine learning techniques includes at least a non-linear regression learning technique, amongst other machine learning techniques; and
the non-linear regression technique includes at least one or more of a random forest and a neural network, amongst other machine learning techniques.
17. The method of claim 16 , wherein:
the predicted performance characteristics are based on at least one or more of each storage device's system configurations and workloads present in the collected telemetry information, amongst other data;
the telemetry information processed by the one or more machine learning techniques is augmented with each storage device's uncollected and validated telemetry information.
18. The method of claim 17 , wherein each storage device's uncollected and validated telemetry information includes at least one or more of each storage device's validated system configurations and validated workloads not present in each storage device's collected telemetry information.
19. The method of claim 11 further comprising:
for one or more sets of storage devices, building one or more system configuration prediction models configured to determine probabilities of each recommended storage device configuration meeting each storage device's SL performance tiers under one or more different workload conditions, wherein each set of storage devices.
20. The method of claim 19 , wherein:
each different workload condition is based on one or more anticipated workloads; and
each of the one or more anticipated workloads is predicted using one or more machine learning techniques to process one or more of: a customer type associated with each requested performance characteristic, the collected telemetry information, and uncollected telemetry information.