Adjusting host quality of service metrics based on storage system performance
A storage system has a QOS recommendation engine that monitors storage system operational parameters and generates recommended changes to host QOS metrics (throughput, bandwidth, and response time requirements) based on differences between the host QOS metrics and storage system operational parameters. The recommended host QOS metrics may be automatically implemented to adjust the host QOS metrics. By reducing host QOS metrics during times where the storage system is experiencing high volumes of workload, it is possible to throttle workload at the host computer rather than requiring the storage system to expend processing resources associated with queueing the workload prior to processing. This can enable the overall throughput of the storage system to increase. When the workload on the storage system is reduced, updated recommended host QOS metrics are provided to enable the host QOS metrics to increase. Historical analysis is also used to generate recommended host QOS metrics.
1. A non-transitory tangible computer readable storage medium having stored thereon a computer program for adjusting host QOS metrics based on storage system performance, the computer program including a set of instructions which, when executed by a computer, cause the computer to perform a method comprising the steps of:
setting host QOS metrics by a host, the host QOS metrics specifying throughput, bandwidth, and response parameters to be achieved by a storage system in connection with servicing a workload from the host on the storage system, the storage system comprising a plurality of storage groups;
receiving the workload by the storage system from the host, the workload being associated with the host QOS metrics;
collecting storage system performance data for each storage group during a time window;
comparing storage system performance data with the host QOS metrics for each storage group during the time window to determine if the storage system performance is compliant with the host QOS metrics;
generating recommended host QOS metrics for each storage group based on the storage system performance data and the host QOS metrics during the time window;
using the recommended host QOS metrics to automatically set the host QOS metrics during the next upcoming time window; and
iterating, for each time window, the steps of:
receiving the workload;
collecting storage system performance data;
comparing storage system performance data with the host QOS metrics;
generating recommended host QOS metrics; and
using the recommended host QOS metrics to automatically set the host QOS metrics during the next upcoming time window;
wherein when the difference between storage system performance and host QOS metrics exceeds a threshold, the step of generating recommend host QOS metrics and using the recommended host QOS metrics to automatically set the host QOS metrics comprises reducing the host QOS metrics to throttle the workload from the host on the storage system;
wherein the step of generating recommended host QOS metrics is implemented, for each storage group, for each time window, using a composite recommendation function;
wherein the host QOS metrics comprise throughput, bandwidth, and response time requirements per storage group;
wherein the composite recommendation function has a first function that determines, for each storage group for each time window, a respective difference between the storage system throughput for that storage group and the host QOS metric for throughput for that storage group;
wherein the composite recommendation function has a second function that determines, for each storage group for each time window, a respective difference between the storage system bandwidth for that storage group and the host QOS metric for bandwidth for that storage group;
wherein the composite recommendation function has a third function that determines, for each storage group for each time window, a respective difference between the storage system response time for that storage group and the host QOS metric for response time for that storage group; and
wherein the composite recommendation function separately weights each of the first, second, and third functions, to enable greater emphasis to be placed on one of the functions of the composite function when determining recommended host QOS metrics.
2. The non-transitory tangible computer readable storage medium of claim 1 , further comprising using the storage system performance data to learn historical storage system usage patterns, and using the historical storage system usage patterns in the step of generating recommended host QOS metrics.
3. The non-transitory tangible computer readable storage medium of claim 1 , wherein the composite recommendation function includes additional functions based on whether prefetch on a cache has been turned on, and CPU usage levels for storage system functions.
4. The non-transitory tangible computer readable storage medium of claim 3 , wherein the storage system functions include data replication to other storage systems, scrubbers, and low priority tasks.
5. A method of adjusting host QOS metrics based on storage system performance, comprising the steps of:
setting host QOS metrics by a host, the host QOS metrics specifying throughput, bandwidth, and response parameters to be achieved by a storage system in connection with servicing a workload from the host on the storage system, the storage system comprising a plurality of storage groups;
receiving the workload by the storage system from the host, the workload being associated with the host QOS metrics;
collecting storage system performance data for each storage group during a time window;
comparing storage system performance data with the host QOS metrics for each storage group during the time window to determine if the storage system performance is compliant with the host QOS metrics;
generating recommended host QOS metrics for each storage group based on the storage system performance data and the host QOS metrics during the time window;
using the recommended host QOS metrics to automatically set the host QOS metrics during the next upcoming time window; and
iterating, for each time window, the steps of:
receiving the workload;
collecting storage system performance data;
comparing storage system performance data with the host QOS metrics;
generating recommended host QOS metrics; and
using the recommended host QOS metrics to automatically set the host QOS metrics during the next upcoming time window;
wherein when the difference between storage system performance and host QOS metrics exceeds a threshold, the step of generating recommend host QOS metrics and using the recommended host QOS metrics to automatically set the host QOS metrics comprises reducing the host QOS metrics to throttle the workload from the host on the storage system;
wherein the step of generating recommended host QOS metrics is implemented, for each storage group, for each time window, using a composite recommendation function;
wherein the host QOS metrics comprise throughput, bandwidth, and response time requirements per storage group;
wherein the composite recommendation function has a first function that determines, for each storage group for each time window, a respective difference between the storage system throughput for that storage group and the host QOS metric for throughput for that storage group;
wherein the composite recommendation function has a second function that determines, for each storage group for each time window, a respective difference between the storage system bandwidth for that storage group and the host QOS metric for bandwidth for that storage group;
wherein the composite recommendation function has a third function that determines, for each storage group for each time window, a respective difference between the storage system response time for that storage group and the host QOS metric for response time for that storage group; and
wherein the composite recommendation function separately weights each of the first, second, and third functions, to enable greater emphasis to be placed on one of the functions of the composite function when determining recommended host QOS metrics.
6. The method of claim 5 , further comprising using the storage system performance data to learn historical storage system usage patterns, and using the historical storage system usage patterns in the step of generating recommended host QOS metrics.
7. A method of adjusting host QOS metrics based on storage system performance, comprising the steps of:
setting host QOS metrics by a host, the host QOS metrics specifying throughput, bandwidth, and response parameters to be achieved by a storage system in connection with servicing a workload from the host on the storage system, the storage system comprising a plurality of storage groups;
receiving the workload by the storage system from the host, the workload being associated with the host QOS metrics;
collecting storage system performance data for each storage group during a time window;
comparing storage system performance data with the host QOS metrics for each storage group during the time window to determine if the storage system performance is compliant with the host QOS metrics;
generating recommended host QOS metrics for each storage group based on the storage system performance data and the host QOS metrics during the time window;
using the recommended host QOS metrics to automatically set the host QOS metrics during the next upcoming time window; and
iterating, for each time window, the steps of:
receiving the workload;
collecting storage system performance data;
comparing storage system performance data with the host QOS metrics;
generating recommended host QOS metrics; and
using the recommended host QOS metrics to automatically set the host QOS metrics during the next upcoming time window;
wherein when the difference between storage system performance and host QOS metrics exceeds a threshold, the step of generating recommend host QOS metrics and using the recommended host QOS metrics to automatically set the host QOS metrics comprises reducing the host QOS metrics to throttle the workload from the host on the storage system;
wherein the step of generating recommended host QOS metrics is implemented, for each storage group, for each time window, using a composite recommendation function;
wherein the host QOS metrics comprise throughput, bandwidth, and response time requirements per storage group;
wherein the composite recommendation function has a first function that determines, for each storage group for each time window, a respective difference between the storage system throughput for that storage group and the host QOS metric for throughput for that storage group;
wherein the composite recommendation function has a second function that determines, for each storage group for each time window, a respective difference between the storage system bandwidth for that storage group and the host QOS metric for bandwidth for that storage group;
wherein the composite recommendation function has a third function that determines, for each storage group for each time window, a respective difference between the storage system response time for that storage group and the host QOS metric for response time for that storage group; and
wherein the composite recommendation function includes additional functions based on whether prefetch on a cache has been turned on, and CPU usage levels for storage system functions.
8. The method of claim 7 , wherein the storage system functions include data replication to other storage systems, scrubbers, and low priority tasks.