Resource monitoring and allocation using proportional-integral-derivative controllers
Techniques for allocating resources in a system may include: monitoring, using a first proportional-integral-derivative (PID) controller, a size of a pool of free shared resources of a first type; responsive to determining the size of the pool of the free shared resources is at least a minimum threshold, providing the size of the pool of free shared resources as an input to a second PID controller; monitoring, using the second PID controller, a total amount of resources of the first type that are available; determining, using the second PID controller and in accordance with one or more resource policies for one or more applications, a deallocation rate or amount; deallocating, using the second PID controller and in accordance with the deallocation rate or amount, resources of the first type; and allocating a least a first of the deallocated resources for use by one of the applications.
1. A method of allocating resources in a system comprising:
monitoring, using a first proportional-integral-derivative (PID) controller, a size of a pool of free shared resources of a first type;
responsive to determining that the size of the pool of the free shared resources of the first type is greater than or equal to a minimum threshold, providing the size of the pool of free shared resources as an input to a second PID controller;
monitoring, using the second PID controller, a total amount of resources of the first type that are available;
determining, using the second PID controller and in accordance with one or more resource policies for one or more applications, a deallocation rate or amount for resources of the first type;
deallocating, using the second PID controller and in accordance with the deallocation rate or amount, one or more resources of the first type; and
allocating a least a first of the one or more resources for use by a first of the one or more applications.
2. The method of claim 1 , wherein the system is a data storage system.
3. The method of claim 1 , wherein the first type of resources is memory.
4. The method of claim 1 , wherein the first type of resources is CPU resources.
5. The method of claim 4 , wherein the pool of free shared CPU resources denotes CPU resources of a single board that are available for use by any application executing on a CPU of the single board.
6. The method of claim 5 , wherein the total amount of resources of the first type that are available denotes a total amount of CPU resources on the single board that are available.
7. The method of claim 6 , wherein the total amount of CPU resources that are available includes first CPU resources of the pool of free shared CPU resources and includes second CPU resources that are allocated for dedicated use by one or more applications executing on one or more CPUs of the single board.
8. The method of claim 7 , wherein the second CPU resources allocated for dedicated use by the one or more applications includes a first portion which are currently not in use by any of the one or more applications.
9. The method of claim 6 , wherein the total amount of CPU resources on the single board that are available denotes a number of CPUs of the single board where each of the number of CPUs currently has a zero-percentage utilization.
10. The method of claim 1 , further comprising:
varying the minimum threshold at each of various points in time in accordance with an expected or predicted change in resource requirements of at least one application that consumes resources of the pool of free shared resources of the first type.
11. A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of allocating resources in a system comprising:
monitoring, using a first proportional-integral-derivative (PID) controller, a size of a pool of free shared resources of a first type;
responsive to determining that the size of the pool of the free shared resources of the first type is greater than or equal to a minimum threshold, providing the size of the pool of free shared resources as an input to a second PID controller;
monitoring, using the second PID controller, a total amount of resources of the first type that are available;
determining, using the second PID controller and in accordance with one or more resource policies for one or more applications, a deallocation rate or amount for resources of the first type;
deallocating, using the second PID controller and in accordance with the deallocation rate or amount, one or more resources of the first type; and
allocating a least a first of the one or more resources for use by a first of the one or more applications.
12. The non-transitory computer readable medium of claim 11 , wherein the system is a data storage system.
13. The non-transitory readable medium of claim 11 , wherein the first type of resources is memory.
14. The non-transitory computer readable medium of claim 11 , wherein the first type of resources is CPU resources.
15. The non-transitory computer readable medium of claim 14 , wherein the pool of free shared CPU resources denotes CPU resources of a single board that are available for use by any application executing on a CPU of the single board.
16. The non-transitory computer readable medium of claim 15 , wherein the total amount of resources of the first type that are available denotes a total amount of CPU resources on the single board that are available.
17. The non-transitory computer readable medium of claim 16 , wherein the total amount of CPU resources that are available includes first CPU resources of the pool of free shared CPU resources and includes second CPU resources that are allocated for dedicated use by one or more applications executing on one or more CPUs of the single board.
18. A data storage system comprising:
a plurality of director boards, wherein each of the plurality of director boards includes a plurality of CPUs configured as a plurality of adapters;
a memory coupled to the plurality of director boards;
a computer readable medium comprising code stored thereon that, when executed, performs a method of allocating resources in the data storage system comprising:
monitoring, using a first proportional-integral-derivative (PID) controller, a size of a pool of free shared CPU resources on a first of the plurality of director boards;
responsive to determining that the size of the pool of the free shared CPU resources of the first director board is greater than or equal to a minimum threshold, providing the size of the pool of free shared CPU resources of the first director board as an input to a second PID controller;
monitoring, using the second PID controller, a total amount of CPU resources of the first board that are available;
determining, using the second PID controller and in accordance with one or more resource policies for one or more applications, a deallocation rate or amount for CPU resources of the first director board;
deallocating, using the second PID controller and in accordance with the deallocation rate or amount, one or more CPU resources of the first director board; and
allocating a least a first of the one or more CPU resources of the first director board for use by a first of the one or more applications.
19. The data storage system of claim 18 , wherein the plurality of adapters includes a host adapter that exchanges data with one or more hosts connected to the data storage system.
20. The data storage system of claim 18 , wherein the plurality of adapters includes a disk adapter that exchanges data with one or more physical storage devices.
21. The data storage system of claim 18 , wherein the plurality of adapters includes a remote adapter that exchanges data with one or more remote data storage systems.
22. The data storage system of claim 18 , wherein each of the plurality of adapters of said each director board are emulated using code that executes on one or more processors of one or more CPU resources of said each director board.