Sequentiality characterization of input/output workloads
Techniques are provided for characterizing and quantifying a sequentiality of workloads using sequentiality profiles and signatures. One exemplary method comprises obtaining telemetry data for an input/output workload; evaluating a distribution over time of sequence lengths for input/output requests in the telemetry data by the input/output workload; and generating a sequentiality profile for the input/output workload to characterize the input/output workload based at least in part on the distribution over time of the sequence lengths. Multiple sequentiality profiles for one or more input/output workloads may be clustered into a plurality of clusters. A sequentiality signature may be generated to represent one or more sequentiality profiles within a given cluster. A performance of data movement policies may be evaluated with respect to the sequentiality signature of the given cluster.
1. A method, comprising:
obtaining telemetry data for an input/output workload of a storage system, wherein the input/output workload comprises a plurality of input/output requests related to data stored by the storage system;
evaluating a distribution over time of sequence lengths for the input/output requests in the telemetry data by the input/output workload; and
generating a sequentiality profile for the input/output workload to characterize the input/output workload based at least in part on the distribution over time of the sequence lengths, wherein the sequentiality profile indicates a probability of each of a plurality of sequence lengths;
clustering a plurality of sequentiality profiles for one or more input/output workloads into a plurality of clusters;
generating a sequentiality signature to represent one or more sequentiality profiles within a given cluster of the plurality of clusters; and
initiating an adjustment of one or more of a prefetch amount and a prefetch trigger condition of the at least one data movement policy by evaluating a quantification of a sequentiality of the input/output workload using the sequentiality signature of the given cluster, wherein the quantification characterizes a degree of the sequentiality of the input/output workload and wherein an amount of the adjustment to the prefetch amount directly correlates with the quantification and an amount of the adjustment to the prefetch trigger condition inversely correlates with the quantification;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.
2. The method of claim 1 , wherein the evaluating the distribution over time of sequence lengths for input/output requests comprises determining if two subsequent input/output requests are sequential.
3. The method of claim 2 , wherein the determining if two subsequent input/output requests are sequential comprises allowing a predefined gap between subsequent input/output request addresses.
4. The method of claim 1 , wherein the evaluating the distribution over time of sequence lengths for input/output requests further comprises sorting the input/output requests by address and discarding repeated accesses to the same address.
5. The method of claim 1 , wherein the evaluating the distribution over time of sequence lengths for subsequent input/output requests employs a clamping value comprising a maximum sequence length.
6. The method of claim 1 , wherein the sequentiality signature is a centroid of the given cluster.
7. The method of claim 1 , further comprising evaluating a quality of one or more of the plurality of clusters based at least in part on one or more of an average intra-cluster deviation of at least one cluster and a silhouette of the plurality of clusters.
8. The method of claim 1 , further comprising evaluating at least one performance metric, wherein the at least one performance metric comprises one or more of a hit ratio, a miss ratio and a pollution ratio with respect to the sequentiality signature of the given cluster.
9. The method of claim 1 , further comprising quantifying wherein the quantification of the sequentiality of the input/output workload is based at least in part on an area under the curve of one or more of the sequentiality profile and the sequentiality signature.
10. The method of claim 1 , wherein the clustering comprises a projection of the plurality of sequentiality profiles from an original space onto another space and a generation of clusters of irregular shape on the another space.
11. The method of claim 1 , wherein the evaluating the distribution over time of sequence lengths comprises discarding sequence lengths shorter than one or more of a predefined value and a precomputed minimal value.
12. The method of claim 1 , wherein the input/output workload comprises a workload for one or more logical storage units.
13. An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured to implement the following steps:
obtaining telemetry data for an input/output workload of a storage system, wherein the input/output workload comprises a plurality of input/output requests related to data stored by the storage system;
evaluating a distribution over time of sequence lengths for the input/output requests in the telemetry data by the input/output workload; and
generating a sequentiality profile for the input/output workload to characterize the input/output workload based at least in part on the distribution over time of the sequence lengths, wherein the sequentiality profile indicates a probability of each of a plurality of sequence lengths;
clustering a plurality of sequentiality profiles for one or more input/output workloads into a plurality of clusters;
generating a sequentiality signature to represent one or more sequentiality profiles within a given cluster of the plurality of clusters; and
initiating an adjustment of one or more of a prefetch amount and a prefetch trigger condition of the at least one data movement policy by evaluating a quantification of a sequentiality of the input/output workload using the sequentiality signature of the given cluster, wherein the quantification characterizes a degree of the sequentiality of the input/output workload and wherein an amount of the adjustment to the prefetch amount directly correlates with the quantification and an amount of the adjustment to the prefetch trigger condition inversely correlates with the quantification.
14. The apparatus of claim 13 , further comprising evaluating a quality of one or more of the plurality of clusters based at least in part on one or more of an average intra-cluster deviation of at least one cluster and a silhouette of the plurality of clusters.
15. The apparatus of claim 13 , wherein the quantification of the sequentiality of the input/output workload is based at least in part on an area under the curve of one or more of the sequentiality profile and the sequentiality signature.
16. A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform the following steps:
obtaining telemetry data for an input/output workload of a storage system, wherein the input/output workload comprises a plurality of input/output requests related to data stored by the storage system;
evaluating a distribution over time of sequence lengths for the input/output requests in the telemetry data by the input/output workload; and
generating a sequentiality profile for the input/output workload to characterize the input/output workload based at least in part on the distribution over time of the sequence lengths, wherein the sequentiality profile indicates a probability of each of a plurality of sequence lengths;
clustering a plurality of sequentiality profiles for one or more input/output workloads into a plurality of clusters;
generating a sequentiality signature to represent one or more sequentiality profiles within a given cluster of the plurality of clusters; and
initiating an adjustment of one or more of a prefetch amount and a prefetch trigger condition of the at least one data movement policy by evaluating a quantification of a sequentiality of the input/output workload using the sequentiality signature of the given cluster, wherein the quantification characterizes a degree of the sequentiality of the input/output workload and wherein an amount of the adjustment to the prefetch amount directly correlates with the quantification and an amount of the adjustment to the prefetch trigger condition inversely correlates with the quantification.
17. The method of claim 1 , wherein the prefetch amount comprises a size of a look-ahead window.
18. The apparatus of claim 13 , wherein the prefetch amount comprises a size of a look-ahead window.
19. The apparatus of claim 13 , further comprising evaluating at least one performance metric, wherein the at least one performance metric comprises one or more of a hit ratio, a miss ratio and a pollution ratio with respect to the sequentiality signature of the given cluster.
20. The non-transitory processor-readable storage medium of claim 16 , further comprising evaluating at least one performance metric, wherein the at least one performance metric comprises one or more of a hit ratio, a miss ratio and a pollution ratio with respect to the sequentiality signature of the given cluster.