Rate limiter state caching
Reduction in or avoidance of latency in a rate limiting process is achieved by the implementation of a cache, on a rate limiting client, storing a current state of a rate liming server. The rate limiting client receives a communication from a source device that is destined to a destination device. The rate limiting client determines the state of rate limiting. If the state of rate limiting indicates that rate limiting is not in place on the rate limiting server, the rate limiting client transmits the communication to the destination device without waiting for a response from the rate limiting server. The rate limiting client queries the rate limiting server as to whether to update the state.
1. A method comprising:
receiving, at a rate limiting client, a communication from a source device that is destined to a destination device;
determining, at the rate limiting client and in response to receiving the communication, a state of rate limiting for a path of the communication;
in response to receiving the communication, transmitting, from the rate limiting client to a rate limiting server, a query as to whether to update the state of rate limiting;
in response to the rate limiting client determining that the state of rate limiting is a permissive state, transmitting the communication to the destination device without waiting for a response from the rate limiting server to the query as to whether to update the state of rate limiting; and
in response to the rate limiting client determining that the state of rate limiting is a strict state, holding the communication at the rate limiting client and waiting for the response from the rate limiting server before determining whether to transmit the communication to the destination device.
2. The method of claim 1 , further comprising,
in response to the rate limiting client determining that the state of rate limiting is the strict state determining whether the response from the rate limiting server indicates that the communication is rate limited or the response from the rate limiting server indicates that the communication remains below a rate limit.
3. The method of claim 2 , further comprising, in response to determining that the response from the rate limiting server indicates that the communication is rate limited:
transmitting a notification to the source device that the communication is rate limited; and
discarding the communication.
4. The method of claim 2 , further comprising, in response to determining that the response from the rate limiting server indicates that the communication remains below the rate limit, transmitting the communication to the destination device.
5. The method of claim 1 , wherein determining the state of rate limiting comprises:
determining a plurality of attributes of the communication;
identifying a cache key corresponding to the plurality of attributes, the cache key identified from a plurality of cache keys; and
determining the state of rate limiting from the cache key.
6. The method of claim 5 , further comprising:
receiving, from the rate limiting server, a broadcast key update; and
responsive to receiving the broadcast key update, updating the state of rate limiting for each of the plurality of cache keys.
7. The method of claim 1 , further comprising updating the state of rate limiting based on the response from the rate limiting server.
8. The method of claim 1 , further comprising:
determining a state of rate limiting for the communication by:
determining attributes associated with the path of the communication;
inputting the attributes into a supervised machine learning model; and
receiving, as output from the supervised machine learning model, the state of rate limiting.
9. The method of claim 1 , wherein determining the state of rate limiting comprises:
receiving, from the rate limiting server, a broadcast key update; and
responsive to receiving the broadcast key update, updating the state of rate limiting for each of a plurality of cache keys referenced in the broadcast key update.
10. A non-transitory computer-readable medium comprising memory with instructions encoded thereon, the instructions, when executed by one or more processors, causing the one or more processors to perform operations, the instructions comprising instructions to:
receive, at a rate limiting client, a communication from a source device that is destined to a destination device;
determine, at the rate limiting client and in response to receiving the communication, a state of rate limiting for a path of the communication;
in response to receiving the communication, transmit, from the rate limiting client to a rate limiting server, a query as to whether to update the state of rate limiting;
in response to the rate limiting client determining that the state of rate limiting is a permissive state, transmit the communication to the destination device without waiting for a response from the rate limiting server to the query as to whether to update the state of rate limiting; and
in response to the rate limiting client determining that the state of rate limiting is a strict state, hold the communication at the rate limiting client and wait for the response from the rate limiting server before determining whether to transmit the communication to the destination device.
11. The non-transitory computer-readable medium of claim 10 , wherein the instructions further comprise to:
in response to the rate limiting client determining that the state of rate limiting is strict, determine whether the response from the rate limiting server indicates that the communication is rate limited or the response from the rate limiting server indicates that the communication remains below a rate limit.
12. The non-transitory computer-readable medium of claim 11 , wherein the instructions further comprise instructions to:
in response to determining that the response from the rate limiting server indicates that the communication is rate limited:
transmit a notification to the source device that the communication is rate limited; and
discard the communication.
13. The non-transitory computer-readable medium of claim 11 , wherein the instructions further comprise instructions to:
in response to determining that the response from the rate limiting server indicates that the communication remains below the rate limit, transmit the communication to the destination device.
14. The non-transitory computer-readable medium of claim 10 , wherein the instructions for determining the state of rate limiting of the rate limiting server comprise instructions to:
determine a plurality of attributes of the communication;
identify a cache key corresponding to the plurality of attributes, the cache key identified from a plurality of cache keys; and
determine the state of rate limiting from the cache key.
15. The non-transitory computer-readable medium of claim 14 , wherein the instructions further comprise instructions to:
receive, from the rate limiting server, a broadcast key update; and
responsive to receiving the broadcast key update, update the state of rate limiting for each of the plurality of cache keys.
16. The non-transitory computer-readable medium of claim 10 , further comprising updating the state of rate limiting based on the response from the rate limiting server.
17. The non-transitory computer-readable medium of claim 10 , wherein the instructions further comprise instructions to:
determine a state of rate limiting state for the communication by:
determining attributes associated with the path of the communication;
inputting the attributes into a supervised machine learning model; and
receiving, as output from the supervised machine learning model, the state of rate limiting.
18. The non-transitory computer-readable medium of claim 10 , wherein the instructions for determining the state of rate limiting comprise instructions to:
receive, from the rate limiting server, a broadcast key update; and
responsive to receiving the broadcast key update, update the state of rate limiting for each of a plurality of cache keys referenced in the broadcast key update.
19. A system comprising:
memory with instructions encoded thereon; and
one or more processors that, when executing the instructions, are caused to perform operations comprising:
receiving, at a rate limiting client, a communication from a source device that is destined to a destination device;
determining, at the rate limiting client and in response to receiving the communication, a state of rate limiting for a path of the communication;
in response to receiving the communication, transmitting, from the rate limiting client to a rate limiting server, a query as to whether to update the state of rate limiting;
in response to the rate limiting client determining that the state of rate limiting is a permissive state, transmitting the communication to the destination device without waiting for a response from the rate limiting server to the query as to whether to update the state of rate limiting; and
in response to the rate limiting client determining that the state of rate limiting is a strict state, holding the communication at the rate limiting client and waiting for the response from the rate limiting server before determining whether to transmit the communication to the destination device.
20. The system of claim 19 , the operations further comprising:
in response to the rate limiting client determining that the state of rate limiting is the strict state determining whether the response of the rate limiting server indicates that the communication is rate limited or the response from the rate limiting server indicates that the communication remains below a rate limit.