Automatically detecting workload type-related information in storage systems using machine learning techniques
Methods, apparatus, and processor-readable storage media for automatically detecting workload type-related information in storage systems using machine learning techniques are provided herein. An example computer-implemented method includes designating workload types associated with a storage system by identifying performance-related characteristics associated with each of the workload types; obtaining historical time series data, associated with the storage system, pertaining to multiple performance-related metrics; partitioning the historical time series data into multiple segments based on value levels of the performance-related metrics; generating supervised datasets by associating, for each of the segments, the value levels of the performance-related metrics with a given one of the designated workload types; training a machine learning model using the supervised datasets; detecting workload types associated with input data from the storage system by processing the input data using the machine learning model; and performing automated actions based on the detected workload types.
1 . A computer-implemented method comprising:
designating multiple workload types associated with at least one storage system, wherein designating the multiple workload types comprises identifying one or more performance-related characteristics associated with each of the multiple workload types;
obtaining historical time series data, associated with the at least one storage system, pertaining to multiple performance-related metrics;
partitioning at least a portion of the historical time series data into multiple segments based at least in part on value levels of the multiple performance-related metrics;
generating one or more supervised datasets by associating, for each of the multiple segments, at least a portion of the value levels of the multiple performance-related metrics with a given one of the designated workload types;
training at least one machine learning model using at least a portion of the one or more supervised datasets in conjunction with multiple gradient boosted decision trees, wherein the at least one machine learning model comprises at least one gradient boosting model;
detecting one or more workload types associated with input data by processing at least a portion of the input data using the at least one machine learning model; and
performing one or more automated actions based at least in part on the one or more detected workload types associated with the input data, wherein performing one or more automated actions comprises:
automatically configuring at least a portion of infrastructure of a given storage system of the at least one storage system by provisioning one or more hardware components suitable for the one or more detected workload types associated with the input data;
further training the at least one machine learning model based at least in part on feedback related to the one or more detected workload types associated with the input data; and
automatically balancing one or more workloads associated with the input data across portions of the at least one storage system based at least in part on the one or more detected workload types associated with the input data;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.
2 . The computer-implemented method of claim 1 , wherein training the at least one machine learning model comprises processing the at least a portion of the one or more supervised datasets using at least one machine learning multi-class classifier algorithm.
3 . The computer-implemented method of claim 1 , wherein detecting one or more workload types associated with the input data comprises determining that one or more particular storage objects are used for one or more of the designated types of workloads.
4 . The computer-implemented method of claim 1 , wherein obtaining historical time series data comprises obtaining historical time series data pertaining to multiple performance-related metrics at multiple component levels of the at least one storage system.
5 . The computer-implemented method of claim 1 , wherein obtaining historical time series data comprises obtaining historical time series data pertaining to multiple performance-related metrics from multiple storage objects, associated with the at least one storage system, that are allocated to at least one of one or more hosts and one or more virtual machines.
6 . The computer-implemented method of claim 1 , wherein the multiple workload types comprise at least two of batch workloads, transactional workloads, analytic workloads, high-performance workloads, and database workloads.
7 . The computer-implemented method of claim 1 , wherein the one or more performance-related characteristics comprise at least one of input-output mix, input-output type, data-metadata mix, block size distribution, file size distribution, and data efficiency appropriateness.
8 . The computer-implemented method of claim 1 , wherein the multiple performance-related metrics comprise metrics pertaining to two or more of read operations, bandwidth, latency, central processing unit (CPU) utilization, input-output size, input-output operations per second, queue, virtual volume latency, requests, packets, bandwidth, read hits, write hits, and free capacity.
9 . A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device:
to designate multiple workload types associated with at least one storage system, wherein designating the multiple workload types comprises identifying one or more performance-related characteristics associated with each of the multiple workload types;
to obtain historical time series data, associated with the at least one storage system, pertaining to multiple performance-related metrics;
to partition at least a portion of the historical time series data into multiple segments based at least in part on value levels of the multiple performance-related metrics;
to generate one or more supervised datasets by associating, for each of the multiple segments, at least a portion of the value levels of the multiple performance-related metrics with a given one of the designated workload types;
to train at least one machine learning model using at least a portion of the one or more supervised datasets in conjunction with multiple gradient boosted decision trees, wherein the at least one machine learning model comprises at least one gradient boosting model;
to detect one or more workload types associated with input data by processing at least a portion of the input data using the at least one machine learning model; and
to perform one or more automated actions based at least in part on the one or more detected workload types associated with the input data, wherein performing one or more automated actions comprises:
automatically configuring at least a portion of infrastructure of a given storage system of the at least one storage system by provisioning one or more hardware components suitable for the one or more detected workload types associated with the input data;
further training the at least one machine learning model based at least in part on feedback related to the one or more detected workload types associated with the input data; and
automatically balancing one or more workloads associated with the input data across portions of the at least one storage system based at least in part on the one or more detected workload types associated with the input data.
10 . The non-transitory processor-readable storage medium of claim 9 , wherein training the at least one machine learning model comprises processing the at least a portion of the one or more supervised datasets using at least one machine learning multi-class classifier algorithm.
11 . The non-transitory processor-readable storage medium of claim 9 , wherein detecting one or more workload types associated with the input data comprises determining that one or more particular storage objects are used for one or more of the designated types of workloads.
12 . The non-transitory processor-readable storage medium of claim 9 , wherein obtaining historical time series data comprises obtaining historical time series data pertaining to multiple performance-related metrics at multiple component levels of the at least one storage system.
13 . The non-transitory processor-readable storage medium of claim 9 , wherein obtaining historical time series data comprises obtaining historical time series data pertaining to multiple performance-related metrics from multiple storage objects, associated with the at least one storage system, that are allocated to at least one of one or more hosts and one or more virtual machines.
14 . An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured:
to designate multiple workload types associated with at least one storage system, wherein designating the multiple workload types comprises identifying one or more performance-related characteristics associated with each of the multiple workload types;
to obtain historical time series data, associated with the at least one storage system, pertaining to multiple performance-related metrics;
to partition at least a portion of the historical time series data into multiple segments based at least in part on value levels of the multiple performance-related metrics;
to generate one or more supervised datasets by associating, for each of the multiple segments, at least a portion of the value levels of the multiple performance-related metrics with a given one of the designated workload types;
to train at least one machine learning model using at least a portion of the one or more supervised datasets in conjunction with multiple gradient boosted decision trees, wherein the at least one machine learning model comprises at least one gradient boosting model;
to detect one or more workload types associated with input data by processing at least a portion of the input data using the at least one machine learning model; and
to perform one or more automated actions based at least in part on the one or more detected workload types associated with the input data, wherein performing one or more automated actions comprises:
automatically configuring at least a portion of infrastructure of a given storage system of the at least one storage system by provisioning one or more hardware components suitable for the one or more detected workload types associated with the input data;
further training the at least one machine learning model based at least in part on feedback related to the one or more detected workload types associated with the input data; and
automatically balancing one or more workloads associated with the input data across portions of the at least one storage system based at least in part on the one or more detected workload types associated with the input data.
15 . The apparatus of claim 14 , wherein training the at least one machine learning model comprises processing the at least a portion of the one or more supervised datasets using at least one machine learning multi-class classifier algorithm.
16 . The apparatus of claim 14 , wherein detecting one or more workload types associated with the input data comprises determining that one or more particular storage objects are used for one or more of the designated types of workloads.
17 . The apparatus of claim 14 , wherein obtaining historical time series data comprises obtaining historical time series data pertaining to multiple performance-related metrics at multiple component levels of the at least one storage system.
18 . The apparatus of claim 14 , wherein obtaining historical time series data comprises obtaining historical time series data pertaining to multiple performance-related metrics from multiple storage objects, associated with the at least one storage system, that are allocated to at least one of one or more hosts and one or more virtual machines.
19 . The apparatus of claim 14 , wherein the multiple workload types comprise at least two of batch workloads, transactional workloads, analytic workloads, high-performance workloads, and database workloads.
20 . The apparatus of claim 14 , wherein the one or more performance-related characteristics comprise at least one of input-output mix, input-output type, data-metadata mix, block size distribution, file size distribution, and data efficiency appropriateness.