Applying a rate limit across a plurality of storage systems

1. A method comprising:

determining whether an amount of I/O operations serviced by a second storage system during a previous predetermined period of time is less than a local rate limit for the second storage system; and

responsive to determining that the amount of I/O operations serviced by the second storage system during the previous predetermined period of time is less than the local rate limit for the second storage system, setting a local rate limit for a first storage system based on the amount of I/O operations serviced by the second storage system during the previous predetermined period of time.

2. The method of claim 1 further comprising:

receiving, by the second storage system from the first storage system, an amount of I/O operations serviced by the first storage system during a previous predetermined period of time;

determining whether the amount of I/O operations serviced by the first storage system during a previous predetermined period of time is less than the local rate limit for the first storage system;

responsive to determining that the amount of I/O operations serviced by the first storage system during the previous predetermined period of time is less than the local rate limit for the first storage system, setting a local rate limit for a next predetermined period of time for the second storage system to the difference between the rate limit for the paired storage systems and the amount of I/O operations serviced by the first storage system during the previous predetermined period of time.

3. The method of claim 2 further comprising:

sending, from the first storage system to the second storage system at a predetermined interval, information describing the amount of I/O operations serviced by the first storage system during a previous predetermined period of time; and

sending, from the second storage system to the first storage system at the predetermined interval, information describing an amount of I/O operations serviced by the second storage system during the previous predetermined period of time.

4. The method of claim 3 further comprising modifying the predetermined interval.

5. The method of claim 2 further comprising wherein the sum of the local rate limit for the first storage system and the local rate limit for the second storage system exceeds the rate limit specifying the cumulative amount of I/O operations that may be serviced by the paired storage systems during the predetermined period of time.

6. The method of claim 1 further comprising:

detecting that the first storage system has not received information describing the amount of I/O operations serviced by the second storage system during the previous predetermined period of time; and

setting the local rate limit for the next predetermined period of time for the first storage system to the rate limit for the paired storage systems.

7. The method of claim 1 further comprising, within a particular storage system, setting the rate limit for a first node within the particular storage system in dependence upon amount of I/O operations serviced by the second node within the particular storage system during a previous predetermined period of time and the local rate limit for the particular storage system.

8. An apparatus including a computer processor and a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:

determining whether an amount of I/O operations serviced by a second storage system during a previous predetermined period of time is less than a local rate limit for the second storage system; and

responsive to determining that the amount of I/O operations serviced by the second storage system during the previous predetermined period of time is less than the local rate limit for the second storage system, setting a local rate limit for a first storage system based on the amount of I/O operations serviced by the second storage system during the previous predetermined period of time.

9. The apparatus of claim 8 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:

receiving, by the second storage system from the first storage system, an amount of I/O operations serviced by the first storage system during a previous predetermined period of time;

determining whether the amount of I/O operations serviced by the first storage system during a previous predetermined period of time is less than the local rate limit for the first storage system;

responsive to determining that the amount of I/O operations serviced by the first storage system during the previous predetermined period of time is less than the local rate limit for the first storage system, setting a local rate limit for a next predetermined period of time for the second storage system to the difference between the rate limit for the paired storage systems and the amount of I/O operations serviced by the first storage system during the previous predetermined period of time.

10. The apparatus of claim 8 , wherein the sum of the local rate limit for each storage system exceeds rate limit for the system of paired storage systems.

11. The apparatus of claim 8 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the step of:

detecting that information has not been received describing the amount of I/O operations serviced by a particular storage system during the previous predetermined period of time; and

assuming that the particular storage system did not service any I/O operations during the previous predetermined period of time.

12. The apparatus of claim 8 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of, within a particular storage system, setting the rate limit for a first node within the particular storage system in dependence upon amount of I/O operations serviced by the a second node within the particular storage system during a previous predetermined period of time and the local rate limit for the particular storage system.

13. A system of paired storage systems, the system of paired storage systems including a plurality of storage systems, each storage system including a computer processor and a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause each storage system to carry out the steps of:

determining whether an amount of I/O operations serviced by a second storage system during a previous predetermined period of time is less than a local rate limit for the second storage system; and

responsive to determining that the amount of I/O operations serviced by the second storage system during the previous predetermined period of time is less than the local rate limit for the second storage system, setting a local rate limit for a first storage system based on the amount of I/O operations serviced by the second storage system during the previous predetermined period of time.

14. The system of paired storage systems of claim 13 , wherein each storage system further includes computer program instructions that, when executed by the computer processor, cause each storage system to carry out the step of sending, to at least one of the other storage systems at a predetermined interval, information describing the amount of I/O operations serviced by the storage system during a previous predetermined period of time.

15. The system of paired storage systems of claim 13 , wherein each storage system further includes computer program instructions that, when executed by the computer processor, cause each storage system to carry out the step of modifying the predetermined interval.

16. The system of paired storage systems of claim 13 , wherein the sum of the local rate limit for each storage system exceeds rate limit for the system of paired storage systems.

17. The system of paired storage systems of claim 13 , wherein each storage system further includes computer program instructions that, when executed by the computer processor, cause each storage system to carry out the steps of:

detecting that the storage system has not received information describing the amount of I/O operations serviced by at least one other storage system during the previous predetermined period of time; and

assuming that the at least one other storage system did not service any I/O operations during the previous predetermined period of time.

18. The system of paired storage systems of claim 13 , wherein the predetermined share of I/O bandwidth for the storage system is different for at least two of the storage systems.

19. The system of paired storage systems of claim 18 , wherein the configurable share of I/O bandwidth for the other storage systems in the system of paired storage systems is equal to the sum of the predetermined share of I/O bandwidth for each other storage system.

20. The system of paired storage systems of claim 13 wherein each storage system further comprises computer program instructions that, when executed by the computer processor, cause each storage system to carry out the step of setting the rate limit for a first node within the particular storage system in dependence upon amount of I/O operations serviced by the a second node within the particular storage system during a previous predetermined period of time and the local rate limit for the particular storage system.