Web client with response latency awareness
Methods and systems for handling web requests with latency awareness are described herein. A system may receive a web request from a web client, and determine, based on an exponential moving average of past response times, whether to allow the web request to be sent out to the server. Based on this determination, the system may send the web request to the server. The system may receive a response to the web request and update the exponential moving average based on the response time associated with the received response. The response may be forwarded back to the web client.
1. A method comprising:
receiving, from a web client at a client device, a first web request;
determining, at the client device, based on an exponential moving average of a plurality of past response times, whether to allow the first web request, wherein an error response is sent to the web client directly from the client device when the first web request is not allowed, wherein the determining the updated exponential moving average is based on a formula, A U =(R−A C )×C+A C , and wherein A U is the updated exponential moving average, R is the response time, A C is the exponential moving average of the plurality of past response times, and C is a smoothing constant;
sending, based on the determination, the first web request from the client device to a server;
receiving, at the client device, from the server, a response to the first web request;
determining, at the client device, a response time associated with the response;
determining, at the client device, based on the response time and the exponential moving average of the plurality of past response times, an updated exponential moving average; and
sending the response to the web client at the client device.
2. The method of claim 1 , wherein the first web request comprises a hypertext transfer protocol (HTTP) request.
3. The method of claim 1 , wherein the sending the first web request comprises initiating a timer, and
wherein the determining the response time comprises:
calculating the response time based on the timer, wherein the timer comprises a timeout value; and
recording the response time.
4. The method of claim 3 , wherein calculating the response time based on the timer comprises:
calculating an elapsed time from a timer start time; and
comparing the elapsed time to the timer timeout value, wherein the timer timeout value is considered as the response time if the elapsed time is more than the timer timeout time.
5. The method of claim 1 , wherein the smoothing constant is based on a formula, C=2÷(N+1), and wherein N is a weight value.
6. The method of claim 1 , wherein the first web request comprises an indication of a category associated with the first web request, and wherein the plurality of past response times are associated with the category.
7. The method of claim 1 , further comprising:
receiving, from the web client and after receiving the first web request, a second web request;
based on the updated exponential moving average, determining at the client device to disallow the second web request from being sent from the web client to the server; and
sending an error response to the web client.
8. The method of claim 7 , wherein the error response comprises a hypertext transfer protocol (HTTP) response status code indicating too many requests.
9. The method of claim 1 , wherein the determining whether to allow the first web request comprises at least one of:
determining to allow the first web request based on a formula, A C +(A C ×C)>T, being satisfied, wherein A C is the exponential moving average of the plurality of past response times, C is a smoothing constant, and T is a timeout value, or
determining to disallow the first web request based on the formula being not satisfied.
10. The method of claim 1 , wherein the determining whether to allow the first web request comprises determining, among a plurality of exponential moving averages associated with respective categories, the exponential moving average associated with the category.
11. The method of claim 1 , wherein the updated exponential moving average is a current exponential moving average.
12. An apparatus comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
receive, from a web client at the apparatus, a first web request;
determine, based on an exponential moving average of a plurality of past response times, whether to allow the first web request, wherein an error response is sent to the web client directly from the apparatus when the first web request is not allowed, wherein the determining the updated exponential moving average is based on a formula, A U =(R−A C )×C+A C , and wherein A U is the updated exponential moving average, R is the response time, A C is the exponential moving average of the plurality of past response times, and C is a smoothing constant;
send, based on the determination, the first web request to a server;
receive, from the server, a response to the first web request;
determine a response time associated with the response;
determine, based on the response time and the exponential moving average of the plurality of past response times, an updated exponential moving average; and
send the response to the web client.
13. The apparatus of claim 12 , wherein the first web request comprises a hypertext transfer protocol (HTTP) request.
14. The apparatus of claim 12 , wherein the smoothing constant is based on a formula, C=2÷(N+1), and wherein N is a weight value.
15. The apparatus of claim 12 , wherein the instructions, when executed by the one or more processors, cause the apparatus to determine whether to allow the first web request by performing at least one of:
determining to allow the first web request based on a formula, A C +(A C ×C)>T, being satisfied, wherein A C is the exponential moving average of the plurality of past response times, C is a smoothing constant, and T is a timeout value, or
determining to disallow the first web request based on the formula being not satisfied.
16. A non-transitory computer-readable medium storing instructions that, when executed, cause:
receiving, from a web client at a client device, a first web request;
determining, at the client device, based on an exponential moving average of a plurality of past response times, whether to allow the first web request, wherein an error response is sent to the web client directly from the client device when the first web request is not allowed, wherein the determining the updated exponential moving average is based on a formula, A U =(R−A C )×C+A C , and wherein A U is the updated exponential moving average, R is the response time, A C is the exponential moving average of the plurality of past response times, and C is a smoothing constant;
sending, based on the determination, the first web request from the client device to a server;
receiving, at the client device from the server, a response to the first web request;
determining, at the client device, a response time associated with the response;
determining, at the client device, based on the response time and the exponential moving average of the plurality of past response times, an updated exponential moving average; and
sending the response to the web client at the client device.
17. The non-transitory computer-readable medium of claim 16 , wherein the first web request comprises a hypertext transfer protocol (HTTP) request.
18. The non-transitory computer-readable medium of claim 16 , wherein the smoothing constant is based on a formula, C=2÷(N+1), and wherein N is a weight value.
19. The non-transitory computer-readable medium of claim 16 , wherein the instructions, when executed, cause the determining whether to allow the first web request by performing at least one of:
determining to allow the first web request based on a formula, A C +(A C ×C)>T, being satisfied, wherein A C is the exponential moving average of the plurality of past response times, C is a smoothing constant, and T is a timeout value, or
determining to disallow the first web request based on the formula being not satisfied.