Load balancing web service by rejecting connections
A load balancer allocates requests to a pool of web servers configured to have low queue capacities. If the queue capacity of a web server is reached, the web server responds to an additional request with a rejection notification to the load balancer, which enables the load balancer to quickly send the rejected request to another web server. Each web server self-monitors its rejection rate. If the rejection rate exceeds a threshold, the number of processes concurrently running on the web server is increased. If the rejection rate falls below a threshold, the number of processes concurrently running on the web server is decreased.
1. A method comprising:
receiving, at a web server of a pool of web servers, a request from a client via a load balancer, the load balancer for allocating requests to the pool of web servers, the web server having a capacity, wherein the capacity is a number of processes that are available to process requests;
responsive to space being available in a queue, placing the request in the queue;
responsive to the queue being full, sending a rejection notification to the load balancer to allow the load balancer to quickly send the rejected request to another web server in the pool;
self-monitoring a rejection rate for requests to the web server;
responsive to the rejection rate exceeding an upper threshold, increasing the capacity of the web server by increasing a number of processes that are able to process requests; and
responsive to the rejection rate being lower than a lower threshold, decreasing the capacity by ending one of the plurality of processes executing on the web server.
2. The method of claim 1 , wherein the web server is configured to have queue capacity of one, wherein queue capacity is a number of requests the queue can hold.
3. A system having a load balancer for allocating requests to a pool of web servers, each of the web servers comprising:
a queue;
a capacity, wherein the capacity is a number of processes that are available to process requests;
a queue manager that receives a request from a client via the load balancer, and responsive to space being available in the queue, places the request in a queue, and responsive to the queue being full, sends a rejection notification to the load balancer to allow the load balancer to quickly send the rejected request to another web server in the pool; and
a process manager that monitors a rejection rate for requests to the web server, and,
responsive to the rejection rate exceeding an upper threshold, increases the capacity by increasing a number of processes that are able to process requests in the web server, and
responsive to the rejection rate being lower than a lower threshold, decreases the capacity by ending one of the plurality of processes executing on the web server.
4. The web server of claim 3 , wherein the queue has a queue capacity of one, wherein queue capacity is a number of requests the queue can hold.