Utilizing machine learning to detect events impacting performance of workloads running on storage systems
A method includes monitoring a given workload running on one or more storage systems to obtain performance data, detecting a given potential performance-impacting event affecting the given workload based at least in part on a given portion of the obtained performance data, and generating a visualization of at least the given portion of the obtained performance data. The method also includes providing the generated visualization as input to a machine learning algorithm, utilizing the machine learning algorithm to classify the given potential performance-impacting event as one of (i) a true positive event affecting performance of the given workload and (ii) a false positive event corresponding to one or more changes in storage resource utilization by the given workload, and modifying provisioning of storage resources of the one or more storage systems responsive to classifying the given potential performance-impacting event as a true positive event affecting performance of the given workload.
1. A method comprising:
monitoring a given workload running on one or more storage systems to obtain performance data associated with the given workload;
detecting a given potential performance-impacting event affecting the given workload based at least in part on a given portion of the obtained performance data;
generating a visualization of at least the given portion of the obtained performance data;
providing the generated visualization as input to a machine learning algorithm;
utilizing the machine learning algorithm to classify the given potential performance-impacting event as one of (i) a true positive event affecting performance of the given workload and (ii) a false positive event corresponding to one or more changes in storage resource utilization by the given workload; and
modifying provisioning of storage resources of the one or more storage systems responsive to classifying the given potential performance-impacting event as a true positive event affecting performance of the given workload;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.
2. The method of claim 1 wherein the performance data comprises a plurality of performance metrics for the given workload over a designated time range.
3. The method of claim 2 wherein the plurality of performance metrics comprise one or more throughput metrics, one or more latency metrics and one or more queue depth metrics.
4. The method of claim 3 wherein the one or more queue depth metrics are derived at least in part from the one or more throughput metrics and the one or more latency metrics.
5. The method of claim 2 wherein the plurality of performance metrics comprises a percentage of input-output (IO) requests of the given workload that are read requests.
6. The method of claim 2 wherein detecting a given potential performance-impacting event affecting the given workload comprises detecting at least one of:
covariance between two or more of a plurality of performance metrics in a time series of the performance data;
correlations between two or more of the plurality of performance metrics in the performance data; and
violation of one or more designated threshold conditions associated with one or more of the plurality of performance metrics in the performance data.
7. The method of claim 1 wherein the machine learning algorithm comprises a neural network architecture.
8. The method of claim 7 wherein the neural network architecture comprises a Convolutional Neural Network (CNN).
9. The method of claim 8 wherein the CNN comprises:
an input layer configured to receive as input an image file comprising the generated visualization;
at least one hidden layer configured to detect features in the generated visualization corresponding to true positive performance impacting events affecting performance of the given workload and false positive events corresponding to one or more changes in storage resource utilization by the given workload; and
an output layer configured to utilize the detected features to classify the given potential performance-impacting event as one of (i) a true positive event affecting performance of the given workload and (ii) a false positive event corresponding to one or more changes in storage resource utilization by the given workload.
10. The method of claim 9 wherein the at least one hidden layer utilizes one or more Rectified Linear Unit (RELU) activation functions.
11. The method of claim 9 wherein the output layer comprises a softmax output layer.
12. The method of claim 1 further comprising, responsive to classifying the given potential performance-impacting event as a false positive event corresponding to one or more changes in storage resource utilization by the given workload, generating and providing an alert to a user associated with the given workload indicating that a change in performance of the given workload is a result of the one or more changes in storage resource utilization by the given workload.
13. The method of claim 12 further comprising generating a recommendation for one or more changes to storage resources allocated to the given workload responsive to the one or more changes in storage resource utilization by the given workload.
14. The method of claim 1 wherein modifying provisioning of storage resources of the one or more storage systems comprises at least one of allocating additional storage resources to the given workload and migrating the given workload from a first set of storage resources to a second set of storage resources.
15. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device:
to monitor a given workload running on one or more storage systems to obtain performance data associated with the given workload;
to detect a given potential performance-impacting event affecting the given workload based at least in part on a given portion of the obtained performance data;
to generate a visualization of at least the given portion of the obtained performance data;
to provide the generated visualization as input to a machine learning algorithm;
to utilize the machine learning algorithm to classify the given potential performance-impacting event as one of (i) a true positive event affecting performance of the given workload and (ii) a false positive event corresponding to one or more changes in storage resource utilization by the given workload; and
to modify provisioning of storage resources of the one or more storage systems responsive to classifying the given potential performance-impacting event as a true positive event affecting performance of the given workload.
16. The computer program product of claim 15 wherein the performance data comprises a plurality of performance metrics for the given workload over a designated time range, and wherein detecting a given potential performance-impacting event affecting the given workload comprises detecting at least one of:
covariance between two or more of a plurality of performance metrics in a time series of the performance data;
correlations between two or more of the plurality of performance metrics in the performance data; and
violation of one or more designated threshold conditions associated with one or more of the plurality of performance metrics in the performance data.
17. The computer program product of claim 15 wherein the machine learning algorithm comprises a Convolutional Neural Network (CNN) architecture comprising:
an input layer configured to receive as input an image file comprising the generated visualization;
at least one hidden layer configured to detect features in the generated visualization corresponding to true positive performance impacting events affecting performance of the given workload and false positive events corresponding to one or more changes in storage resource utilization by the given workload; and
an output layer configured to utilize the detected features to classify the given potential performance-impacting event as one of (i) a true positive event affecting performance of the given workload and (ii) a false positive event corresponding to one or more changes in storage resource utilization by the given workload.
18. An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured:
to monitor a given workload running on one or more storage systems to obtain performance data associated with the given workload;
to detect a given potential performance-impacting event affecting the given workload based at least in part on a given portion of the obtained performance data;
to generate a visualization of at least the given portion of the obtained performance data;
to provide the generated visualization as input to a machine learning algorithm;
to utilize the machine learning algorithm to classify the given potential performance-impacting event as one of (i) a true positive event affecting performance of the given workload and (ii) a false positive event corresponding to one or more changes in storage resource utilization by the given workload; and
to modify provisioning of storage resources of the one or more storage systems responsive to classifying the given potential performance-impacting event as a true positive event affecting performance of the given workload.
19. The apparatus of claim 18 wherein the performance data comprises a plurality of performance metrics for the given workload over a designated time range, and wherein detecting a given potential performance-impacting event affecting the given workload comprises detecting at least one of:
covariance between two or more of a plurality of performance metrics in a time series of the performance data;
correlations between two or more of the plurality of performance metrics in the performance data; and
violation of one or more designated threshold conditions associated with one or more of the plurality of performance metrics in the performance data.
20. The apparatus of claim 18 wherein the machine learning algorithm comprises a Convolutional Neural Network (CNN) architecture comprising:
an input layer configured to receive as input an image file comprising the generated visualization;
at least one hidden layer configured to detect features in the generated visualization corresponding to true positive performance impacting events affecting performance of the given workload and false positive events corresponding to one or more changes in storage resource utilization by the given workload; and
an output layer configured to utilize the detected features to classify the given potential performance-impacting event as one of (i) a true positive event affecting performance of the given workload and (ii) a false positive event corresponding to one or more changes in storage resource utilization by the given workload.