Techniques for dynamic rate-limiting
Systems and methods provide techniques for dynamic rate-limiting, such as techniques that utilize one or more of asynchronous rate-limiting, context-aware rate-limiting, and cost-aware rate-limiting. In one example, a method for asynchronous rate-limiting includes the steps of receiving a rate-limiting request for a service application; extracting one or more policy-defining parameters from the rate-limiting request; querying a local cache storage medium associated with the rate-limit decision node to identify one or more local rate-limiting policies associated with the rate-limiting request; determining, based on the one or more policy-defining parameters and the one or more local rate-limiting policies, a rate-limiting decision for the rate-limiting request; and transmitting the rate-limiting decision to the service application in response to the rate-limiting request.
1. An apparatus for asynchronous rate-limiting at a rate-limit decision node, the apparatus comprising at least one processor and at least one non-transitory memory comprising program code, wherein the at least one non-transitory memory and the program code are configured to, with the at least one processor, cause the apparatus to perform operations configured to at least:
receive a rate-limiting request for a service application;
extract one or more policy-defining parameters from the rate-limiting request;
query a local cache storage medium associated with the rate-limit decision node to identify one or more local rate-limiting policies associated with the rate-limiting request, wherein the local cache storage medium is configured to receive one or more asynchronous configuration data updates to the one or more local rate-limiting policies from a rate-limit synchronization server, and further wherein the rate-limit decision node is configured to update the one or more local rate-limiting policies based on the one or more asynchronous configuration data updates;
determine, based on the one or more policy-defining parameters and the one or more local rate-limiting policies, a rate-limiting decision for the rate-limiting request; and
transmit the rate-limiting decision to the service application in response to the rate-limiting request.
2. The apparatus of claim 1 , wherein determining the rate-limiting decision comprises:
for each local rate-limiting policy of the one or more local rate-limiting policies: determining a related subset of the one or more policy-defining parameters for the local rate-limiting policy,
querying the local cache storage medium based on the related subset to obtain a local rate-limiting counter for the rate-limiting request with respect to the local rate-limiting policy, and
determining a per-policy rate-limiting determination for the local rate-limiting policy based on the local rate-limiting counter and the local rate-limiting policy; and
determining the rate-limiting decision based on each per-policy rate-limiting determination for a local rate-limiting policy of the one or more local rate-limiting policies.
3. The apparatus of claim 2 , wherein determining the per-policy rate-limiting determination for a local rate-limiting policy of the one or more local rate-limiting policies comprises:
determining whether the local rate-limiting counter satisfies the local rate-limiting policy;
in response to determining that the local rate-limiting counter satisfies the local rate-limiting policy, determining an affirmative per-policy rate-limiting determination; and
in response to determining that the local rate-limiting counter fails to satisfy the local rate-limiting policy, determining a negative per-policy rate-limiting determination.
4. The apparatus of claim 3 , wherein the at least one non-transitory memory and the program code are further configured to, with the at least one processor, cause the apparatus to perform operations configured to at least:
in response to determining that the local rate-limiting counter satisfies the local rate-limiting policy, for each local rate-limiting policy of the one or more local rate-limiting policies, update the local rate-limiting counter in accordance with a counter update measure.
5. The apparatus of claim 4 , wherein the counter update measure is determined based on an estimated resource cost of the rate-limiting request for the service application.
6. The apparatus of claim 4 , wherein the at least one non-transitory memory and the program code are further configured to, with the at least one processor, cause the apparatus to perform operations configured to at least:
subsequent to transmitting the rate-limiting decision, transmitting the local rate-limiting counter and a synchronized local rate-limiting counter to the rate-limit synchronization server.
7. The apparatus of claim 6 , wherein the rate-limit synchronization server is configured to:
determine whether the synchronized local rate-limiting counter exceeds the local rate-limiting counter;
in response to determining that the synchronized local rate-limiting counter exceeds the local rate-limiting counter, update a global rate-limiting counter for the rate-limiting request with respect to the local rate-limiting policy based on the local rate-limiting counter; and
in response to determining that the synchronized local rate-limiting counter fails to exceed the local rate-limiting counter, update the global rate-limiting counter for the rate-limiting request with respect to the local rate-limiting policy based on a measure of deviation of the local rate-limiting counter and the synchronized local rate-limiting counter.
8. The apparatus of claim 1 , wherein:
the local cache storage medium is configured to maintain local rate-limiting configuration data comprising the one or more local rate-limiting policies, and
the rate-limit decision node is configured to update the local rate-limiting configuration data based on the one or more asynchronous configuration data updates.
9. The apparatus of claim 8 , wherein the one or more asynchronous configuration data updates are configured to be transmitted to the local cache storage medium by the rate-limit synchronization server at a configuration data update propagation time that is subsequent to a local query time associated with identifying the one or more local rate-limiting policies by the rate-limit decision node.
10. The apparatus of claim 9 , wherein the configuration data update propagation time is a counter data update propagation time.
11. The apparatus of claim 8 , wherein:
the local rate-limiting configuration data comprises local rate-limiting policy data comprising the one or more local rate-limiting policies and local rate-limiting counter data, and
the one or more asynchronous configuration data updates comprise one or more asynchronous policy data updates associated with the local rate-limiting policy data and one or more asynchronous counter data updates associated with the local rate-limiting counter data.
12. The apparatus of claim 11 , wherein:
the one or more asynchronous policy data updates are determined based on global rate-limiting policy data maintained by the rate-limit synchronization server, and
the one or more asynchronous counter data updates are determined based on global rate-limiting counter data maintained by the rate-limit synchronization server.
13. The apparatus of claim 12 , wherein:
the global rate-limiting policy data is stored on a hard disk of the rate-limit synchronization server, and
the global rate-limiting counter data is stored on a main memory of the rate-limit synchronization server.
14. The apparatus of claim 12 , wherein:
the global rate-limiting policy data is stored as a relational database table, and
the global rate-limiting counter data is stored as a collection of key-value pairs.
15. The apparatus of claim 1 , wherein:
the rate-limiting request is associated with a tenant identifier, a user identifier, and a service identifier, and
the one or more policy-defining parameters comprise one or more tenant-related parameters associated with the tenant identifier, one or more user-related parameters associated with the user identifier, and one or more request-related parameters associated with the service identifier.
16. The apparatus of claim 1 , wherein each local rate-limiting policy of the one or more local rate-limiting policies is characterized by:
a parameter descriptor identifying parameter types for a related subset of the one or more policy-defining parameters that is associated with the local rate-limiting policy,
a rate limit threshold value descriptor,
a rate limit threshold measurement unit descriptor,
a rate-limiting technique descriptor,
a target path descriptor, and
a fail mode descriptor.
17. The apparatus of claim 1 , wherein transmitting the rate-limiting decision is configured to be performed within a decision latency period from receiving the rate-limiting request.
18. The apparatus of claim 17 , wherein the decision latency period is one microsecond.
19. A computer-implemented method for asynchronous rate-limiting at a rate-limit decision node, the computer-implemented method comprising:
receiving a rate-limiting request for a service application;
extracting one or more policy-defining parameters from the rate-limiting request;
querying a local cache storage medium associated with the rate-limit decision node to identify one or more local rate-limiting policies associated with the rate-limiting request, wherein the local cache storage medium is configured to receive one or more asynchronous configuration data updates from a rate-limit synchronization server, and further wherein the rate-limit decision node is configured to update the one or more local rate-limiting policies based on the one or more asynchronous configuration data updates;
determining, based on the one or more policy-defining parameters and the one or more local rate-limiting policies, a rate-limiting decision for the rate-limiting request; and
transmitting the rate-limiting decision to the service application in response to the rate-limiting request.
20. A computer program product for asynchronous rate-limiting at a rate-limit decision node, the computer program product comprising at least one non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions configured to:
receive a rate-limiting request for a service application;
extract one or more policy-defining parameters from the rate-limiting request;
query a local cache storage medium associated with the rate-limit decision node to identify one or more local rate-limiting policies associated with the rate-limiting request, wherein the local cache storage medium is configured to receive one or more asynchronous configuration data updates from a rate-limit synchronization server, and further wherein the rate-limit decision node is configured to update the one or more local rate-limiting policies based on the one or more asynchronous configuration data updates;
determine, based on the one or more policy-defining parameters and the one or more local rate-limiting policies, a rate-limiting decision for the rate-limiting request; and
transmit the rate-limiting decision to the service application in response to the rate-limiting request.