System and method for regulating host IOs and internal background operations in a storage system
A method, computer program product, and computer system for monitoring host IO latency. It may be identified that a rate of the host IO latency is at a one of a plurality of levels. At least one of a rate of background IOs and a rate of host IOs may be regulated based upon, at least in part, the rate of the host IO latency being at the one of the plurality of levels.
1. A computer-implemented method comprising:
monitoring host input/output (IO) latency associated with processing host IOs received from host computing devices, on a storage system;
identifying that a rate of the host IO latency is at one of a plurality of levels; and
regulating a rate of background IOs and a rate of host IOs of the storage system based upon, at least in part, the rate of the host IO latency being at the one of the plurality of levels, wherein the background IOs include IOs generated by internal background operations associated with the storage system, wherein regulating the rate of background IOs includes regulating a number of tokens distributed to a background operation scheduler and wherein regulating the rate of host IOs includes regulating a number of tokens distributed to a host IO scheduler.
2. The computer-implemented method of claim 1 wherein the rate of the background IOs is regulated based on a deviation around normalized mean calibrated host IO latency when the rate of the host IO latency is at a first level of the one of the plurality of levels.
3. The computer-implemented method of claim 2 wherein the rate of the background IOs is regulated based on a background application performing the background IOs when the rate of the host IO latency is at a second level of the one of the plurality of levels.
4. The computer-implemented method of claim 3 wherein the rate of the background IOs for all background applications performing the background IOs is set to a minimum rate when the rate of the host IO latency is at a third level of the one of the plurality of levels.
5. The computer-implemented method of claim 4 wherein the rate of the host IOs is regulated when the rate of the host IO latency is at a fourth level of the one of the plurality of levels.
6. The computer-implemented method of claim 1 wherein the rate of one of the background IOs and the rate of host IOs is regulated based on a Quality of Service metric.
7. The computer-implemented method of claim 1 further comprising determining a calibration latency of the host IO latency.
8. A computer program product residing on a non-transitory computer readable storage medium having a plurality of instructions stored thereon which, when executed across one or more processors, causes at least a portion of the one or more processors to perform operations comprising:
monitoring host input/output (IO) latency associated with processing host IOs received from host computing devices, on a storage system;
identifying that a rate of the host IO latency is at one of a plurality of levels; and
regulating a rate of background IOs and a rate of host IOs of the storage system based upon, at least in part, the rate of the host IO latency being at the one of the plurality of levels, wherein the background IOs include IOs generated by internal background operations associated with the storage system, wherein regulating the rate of background IOs includes regulating a number of tokens distributed to a background operation scheduler and wherein regulating the rate of host IOs includes regulating a number of tokens distributed to a host IO scheduler.
9. The computer program product of claim 8 wherein the rate of the background IOs is regulated based on a deviation around normalized mean calibrated host IO latency when the rate of the host IO latency is at a first level of the one of the plurality of levels.
10. The computer program product of claim 9 wherein the rate of the background IOs is regulated based on a background application performing the background IOs when the rate of the host IO latency is at a second level of the one of the plurality of levels.
11. The computer program product of claim 10 wherein the rate of the background IOs for all background applications performing the background IOs is set to a minimum rate when the rate of the host IO latency is at a third level of the one of the plurality of levels.
12. The computer program product of claim 11 wherein the rate of the host IOs is regulated when the rate of the host IO latency is at a fourth level of the one of the plurality of levels.
13. The computer program product of claim 8 wherein the rate of one of the background IOs and the rate of host IOs is regulated based on a Quality of Service metric.
14. The computer program product of claim 8 wherein the operations further comprise determining a calibration latency of the host IO latency.
15. A computing system including one or more processors and one or more memories configured to perform operations comprising:
monitoring host input/output (IO) latency associated with processing host IOs received from host computing devices, on a storage system;
identifying that a rate of the host IO latency is at one of a plurality of levels; and
regulating a rate of background IOs and a rate of host IOs of the storage system based upon, at least in part, the rate of the host IO latency being at the one of the plurality of levels, wherein the background IOs include IOs generated by internal background operations associated with the storage system, wherein regulating the rate of background IOs includes regulating a number of tokens distributed to a background operation scheduler and wherein regulating the rate of host IOs includes regulating a number of tokens distributed to a host IO scheduler.
16. The computing system of claim 15 wherein the rate of the background IOs is regulated based on a deviation around normalized mean calibrated host IO latency when the rate of the host IO latency is at a first level of the one of the plurality of levels.
17. The computing system of claim 16 wherein the rate of the background IOs is regulated based on a background application performing the background IOs when the rate of the host IO latency is at a second level of the one of the plurality of levels.
18. The computing system of claim 17 wherein the rate of the background IOs for all background applications performing the background IOs is set to a minimum rate when the rate of the host IO latency is at a third level of the one of the plurality of levels.
19. The computing system of claim 18 wherein the rate of the host IOs is regulated when the rate of the host IO latency is at a fourth level of the one of the plurality of levels.
20. The computing system of claim 15 wherein the rate of one of the background IOs and the rate of host IOs is regulated based on a Quality of Service metric, wherein the operations further comprise determining a calibration latency of the host IO latency.