Storage rate limiting for information handling system with multiple storage controllers
An I/O management method includes determining local target I/O rates for a plurality of storage controllers across which an I/O stream is distributed. The local target I/O rates may reflect a target latency corresponding to a QoS attribute associated with the I/O stream. A cumulative target I/O rate may be determined based on the local target I/O rates. A token bucket rate may be calculated for each applicable storage controller based on the local target I/O rates. The storage controllers may implement token bucket algorithms in accordance with the token bucket rates to enforce the QoS for the I/O stream. The local target I/O rates may be determined periodically based on measured data indicating actual latency and traffic. A queue depth is determined based on the measured values of latency and traffic. The queue depth is then used with the desired value of latency to obtain the target I/O rates.
1. A method of managing I/O in an information handling system, the method comprising:
determining local target I/O rates for a particular I/O stream in accordance with a target latency for the particular I/O stream, wherein the local target I/O rates include a local target I/O rate for each storage controller in a plurality of storage controllers associated with the particular I/O stream, and wherein the determining of the local target I/O rates includes, for each storage controller in the plurality of storage controllers:
obtaining a measured latency, wherein the measured latency is indicative of recent actual latency associated with the particular I/O stream and the storage controller;
obtaining a measured I/O rate associated with the particular I/O stream and the storage controller;
calculating a current queue depth in accordance with the measured latency and the measured I/O rate; and
determining the local target I/O rate based on the current queue depth and the target latency;
determining a cumulative measured I/O rate from the measured I/O rates and a cumulative target I/O rate from the local target I/O rates;
based on the local target I/O rates, the cumulative measure I/O rate, and the cumulative target I/O rate, calculating token bucket rates including a token bucket rate corresponding to each storage controller; and
configuring each storage controller to:
maintain a token bucket in accordance with the corresponding token bucket rate;
perform pending I/O operations of the particular I/O stream subject to one or more token bucket criteria;
responsive to detecting two or more logical volumes with pending I/O operations from the particular I/O stream, randomly identifying one of the two or more logical volumes as a selected volume; and
issue a next token to the selected volume, wherein tokens correspond to I/O operations and wherein the one or more criteria include a minimum token criteria requiring a positive token bucket.
2. The method of claim 1 , wherein the particular I/O stream comprises an I/O stream associated with a particular class of service.
3. The method of claim 1 , wherein the particular I/O stream comprises I/O operations associated with a particular host process.
4. The method of claim 1 , wherein the particular I/O stream comprises I/O operations addressing a particular logical volume.
5. The method of claim 1 , wherein the particular I/O stream comprises I/O operations addressing a particular logical volume group comprising a user-designated group of two or more logical volumes.
6. The method of claim 1 , wherein determining the cumulative target I/O rate comprises summing the local target I/O rates.
7. The method of claim 6 , wherein calculating a local token bucket rate comprises multiplying a the local target I/O rate by a ratio of a cumulative actual I/O rate to the cumulative target I/O rate.
8. An information handling system, comprising:
a processor;
a non-transient computer readable medium including processor executable I/O management instructions that, when executed by the processor, cause the system to perform I/O management operations comprising:
determining local target I/O rates for a particular I/O stream in accordance with a target latency for the particular I/O stream, wherein the local target I/O rates include a local target I/O rate for each storage controller in a plurality of storage controllers associated with the particular I/O stream, and wherein the determining of the local target I/O rates includes, for each storage controller in the plurality of storage controllers:
obtaining a measured latency, wherein the measured latency is indicative of recent actual latency associated with the particular I/O stream and the storage controller;
obtaining a measured I/O rate associated with the particular I/O stream and the storage controller;
calculating a current queue depth in accordance with the measured latency and the measured I/O rate; and
determining the local target I/O rate based on the current queue depth and the target latency;
determining a cumulative target I/O rate from the local target I/O rates;
based on the local target I/O rates and the cumulative target I/O rate, calculating token bucket rates including a token bucket rate corresponding to each storage controller; and
configuring each storage controller to:
maintain a token bucket in accordance with the corresponding token bucket rate;
perform pending I/O operations of the particular I/O stream subject to one or more token bucket criteria;
responsive to detecting pending I/O operations of the particular I/O stream associated with two or more logical volumes, randomly identifying one of the two or more logical volumes as a current volume; and
issue a next token to the current volume, wherein tokens correspond to I/O operations and wherein the one or more criteria include a minimum token criteria requiring a positive token bucket.
9. The information handling system of claim 8 , wherein the particular I/O stream comprises an I/O stream associated with a particular class of service.
10. The information handling system of claim 8 , wherein the particular I/O stream comprises I/O operations associated with a particular host process.
11. The information handling system of claim 8 , wherein the particular I/O stream comprises I/O operations addressing a particular logical volume.
12. The information handling system of claim 8 , wherein the particular I/O stream comprises I/O operations addressing a particular logical volume group comprising a user-designated group of two or more logical volumes.
13. The information handling system of claim 8 , wherein determining the cumulative target I/O rate comprises summing the local target I/O rates.
14. The information handling system of claim 8 , wherein calculating a local token bucket rate comprises multiplying a the local target I/O rate by a ratio of a cumulative actual I/O rate to the cumulative target I/O rate.