Dynamic capacity management of multiple parallel-connected computing resources
A dynamic capacity management policy for multi-paralleled computing resources (e.g., application servers, virtual application servers, etc.) that includes one or more of a state-change component, a load-balancing component, and a robustness-control component. The state-change component delays the release (e.g., powering down of a physical server, removal from a virtual-server lease, etc.) of each computing resource for a set amount of time. The load-balancing component can work in conjunction with the state-change component to reduce the number of idle computing resources by distributing incoming requests in a manner that keeps the already-processing computing resources as full of requests as possible. The robustness-control component scales capacity as a function of the current number of requests within the system of computing resources to account for variations other than request rate, such as request size, reduced processor frequency, network slowdowns, etc., that affect processing capacity.
1. A method of controlling a plurality of computing resources each having a lower-setup-cost state and a higher-setup-cost state, comprising:
arranging the plurality of computing resources so that each is capable of processing at least a share of an incoming request stream;
controlling each of the plurality of computing resources so that each switches from the lower-setup-cost state to the higher-setup-cost state as a function of a timing-out of a state-change delay timer that is initiated when that computing resource is idled; and
determining a minimum number, k.sub.reqd, of the plurality of computing resources to be in the lower-setup-cost state and overriding ones of the state-change delay timers as needed to keep the minimum number of the plurality of computing resources in the lower-setup-cost state;
wherein:
said determining includes determining k.sub.reqd as a function of a total number of requests currently distributed among the plurality of computing resources and further as a function of a packing factor and a number, k, of the plurality of computing resources currently in the lower-setup-cost state.
2. A method according to claim 1 , wherein: k.sub.reqd=k(p.sub.srv/p.sub.ref) where: p.sub.srv=an average request size multiplied by a request rate into one of the plurality of computing resources, as based on k and the total number of requests currently distributed among the plurality of computing resources, and p.sub.ref=the packing factor multiplied by the average request size.
3. A method according to claim 1 , further comprising setting each state-change delay timer as a function of an arrival rate of the incoming request stream.
4. A processing system for processing an incoming request stream, comprising:
a plurality of computing resources each capable of processing at least a share of the incoming request stream and having a lower-setup-cost state and a higher-setup-cost state;
a load balancer comprising at least one processor designed and configured to distribute new arrivals within the incoming request stream among said plurality of computing resources;
a state-change delay timer for each of said plurality of computing resources;
wherein:
said state-change delay timer is designed and configured to start running as a function of the corresponding one of said plurality of computing resources becoming idle;
and the corresponding one of said plurality of computing resources is switched from said lower-setup-cost state to said higher-setup-cost state as a function of a timing-out of said state-change delay timer;
and a robustness controller designed and configured to determine a minimum number, k.sub.reqd, of said plurality of computing resources to be in said lower-setup-cost state and override ones of said state-change delay timers as needed to keep the minimum number of said plurality of computing resources in said lower-setup-cost state;
wherein:
said robustness controller is designed and configured to determine k.sub.reqd as a function of a total number of requests currently distributed among said plurality of computing resources and further as a function of a packing factor and a number, k, of said plurality of computing resources currently in said lower-setup-cost state.
5. A system according to claim 4 , wherein: k.sub.reqd=k(p.sub.srv/p.sub.ref) where: p.sub.srv=an average request size multiplied by a request rate into one of said plurality of computing resources, as based on k and the total number of requests currently distributed among said plurality of computing resources, and p.sub.ref=the packing factor multiplied by the average request size.
6. A system according to claim 4 , further comprising a timer controller in operative communication with each of said plurality of computing resources and designed and configured to set each state-change delay timer as a function of an arrival rate of the incoming request stream.
7. A machine readable storage medium containing machine-executable instructions for controlling a plurality of computing resources in processing an incoming request stream, wherein each of the plurality of computing resources has a lower-setup-cost state and a higher-setup-cost state, said machine-executable instructions comprising:
a first set of machine-executable instructions for distributing new arrivals in the incoming request stream as a function of a packing factor;
wherein each of the plurality of computing resources has a state-change delay timer and said machine-executable instructions further includes machine-executable instructions for determining a minimum number, k.sub.reqd, of the plurality of computing resources to be in the lower-setup-cost state and overriding ones of the state-change delay timers as needed to keep the minimum number of the plurality of computing resources in the lower-setup-cost state;
and wherein:
said machine-executable instructions further includes machine-executable instructions for determining k.sub.reqd as a function of a total number of requests currently distributed among the plurality of computing resources and further as a function of a packing factor and a number, k, of the plurality of computing resources currently in the lower-setup-cost state.
8. A machine-readable storage medium according to claim 7 , wherein said machine-executable instructions further includes machine-executable instructions for executing the equation k.subseqd=k(p.sub.srv/p.sub.ref) where: p.sub.srv=an average request size multiplied by a request rate into one of the plurality of computing resources, as based on k and the total number of requests currently distributed among the plurality of computing resources, and p.sub.ref=the packing factor multiplied by the average request size.
9. A machine-readable storage medium according to claim 7 , wherein each of the plurality of computing resources has a state-change delay timer and said machine-executable instructions further include machine-executable instructions for setting each state-change delay timer as a function of an arrival rate of the incoming request stream.