SERVER CONTROLLED ADAPTIVE BACK OFF FOR OVERLOAD PROTECTION USING INTERNAL ERROR COUNTS
Embodiments of the present invention relate to server controlled adaptive back off for overload protection. The server controls a back off period for each request, which indicates a retry time of when a request should be resent to the server. This back off approach relies on the server since the server has much more accurate information available on which to make back off decisions. The server changes the retry time based on how busy it is and its ability to handle the current load and/or its downstream dependent systems. This back off approach increases server stability during a very high load by spreading the load out over a longer time period. The server is able to turn a traffic spike into a constant load, which is easier and more efficient for the server to handle.
1 . A non-transitory computer-readable medium storing instructions that, when executed by a computing device, cause the computing device to perform a method, the method comprising:
hosting at least one service;
communicatively coupling with a first end-user device;
receiving a request from the first end-user device for the at least one service;
controlling a back off period of the first end-user device by determining a retry time that is specific to the request from the first end-user device; and
relaying the retry time to the first end-user device.
2 . The non-transitory computer-readable medium of claim 1 , wherein the retry time is based at least on a function of an internal error rate, wherein the internal error rate is observed over a time period.
3 . The non-transitory computer-readable medium of claim 2 , wherein the internal error rate is associated with a number of requests that have been rejected within the time period.
4 . The non-transitory computer-readable medium of claim 2 , wherein the internal error rate is observed on a per service basis.
5 . The non-transitory computer-readable medium of claim 1 , wherein the retry time is based on a function of an error rate observed from downstream systems.
6 . The non-transitory computer-readable medium of claim 1 , wherein the retry time is based on a function a number of pending downstream events.
7 . The non-transitory computer-readable medium of claim 1 , wherein the retry time is based on a priority access associated with a user of the first end-user device.
8 . The non-transitory computer-readable medium of claim 1 , wherein the method further includes receiving, after the retry time has passed, the request for the at least one service resent from the first end-user device.
9 . The non-transitory computer-readable medium of claim 8 , wherein the method further includes processing the resent request.
10 . The non-transitory computer-readable medium of claim 8 , wherein the method further includes repeating the step of controlling a back off period and the step of relaying the retry time.
11 . The non-transitory computer-readable medium of claim 1 , wherein the method further includes receiving a request for the at least one service from a second end-user device at substantially the same time as the request for the at least one service from the first end-user device is received, wherein a retry time determined for the request from the second end-user device is different from the retry time determined for the request from the first end-user device.
12 . The non-transitory computer-readable medium of claim 1 , wherein the method further includes receiving a request for the at least one service from a second end-user device after receiving the request for the at least one service from the first end-user device, wherein a retry time determined for the request from the second end-user device is shorter than the retry time determined for the request from the first end-user device.
13 . The non-transitory computer-readable medium of claim 1 , wherein the method further includes receiving a request for the at least one service from a second end-user device after receiving the request for the at least one service from the first end-user device, wherein a retry time determined for the request from the second end-user device is longer than the retry time determined for the request from the first end-user device.
14 . A non-transitory computer-readable medium storing instructions that, when executed by a computing device, cause the computing device to perform a method, the method comprising:
receiving a plurality of requests from end-user devices that are communicatively coupled with the computing device;
based on a function of an internal error rate, determining a retry time for a first subset of the end-user devices;
informing the first subset of the end-user devices of the retry time; and
processing corresponding requests from a second subset of the end-user devices.
15 . The non-transitory computer-readable medium of claim 14 , wherein the internal rate is observed on a per service basis.
16 . The non-transitory computer-readable medium of claim 14 , wherein the retry time adjusts to computing device overloads and recoveries.
17 . The non-transitory computer-readable medium of claim 14 , wherein corresponding requests from the first subset of the end-user devices and the corresponding requests from the second subset of the end-user devices are for the same service.
18 . The non-transitory computer-readable medium of claim 14 , wherein corresponding requests from a third subset of the end-user devices are for a service that is different from a service that the first subset of the end-user devices is requesting, wherein the method further includes processing the corresponding requests from the third subset of the end-user devices prior to processing corresponding requests from the first subset of the end-user devices.
19 . The non-transitory computer-readable medium of claim 14 , wherein the method includes turning a traffic spike into a constant load.
20 . A computing device comprising:
a system load during a traffic spike;
a network interface for communicatively coupling with at least one end-user device to receive a request; and
a non-transitory computer-readable medium storing instructions to implement:
a counter that counts a number of errors that have occurred within a time period; and
a server controlled adaptive back off module that adjusts a retry time based on an error rate over the time period, wherein the retry time is relayed to the at least one end-user device such that the system load is spread over time.
21 . The computing device of claim 20 , wherein the network interface receives the request resent from the at least one end-user after the retry time has passed.
22 . The computing device of claim 20 , wherein the retry time calculated at a first point in time is longer than the retry time calculated at a second point in time subsequent the first point in time.
23 . The computing device of claim 20 , wherein the retry time calculated at a first point in time is shorter than the retry time calculated a second point in time subsequent the first point in time.
24 . The computing device of claim 20 , wherein the error rate is observed across all services hosted by the computing device.
25 . The computing device of claim 20 , wherein the error rate is observed on a per service basis.
26 . The computing device of claim 20 , wherein the retry time is based on a priority access associated with a user of the at least one end-user device.
27 . The computing device of claim 20 , wherein the server controlled adaptive back off module influences how end-user devices that is communicatively coupled with the computing device behave, wherein each influence is different.