System and method for statistically distributed rate limiting of application programming interface traffic
Disclosed herein are system, method, and computer program product embodiments for implementing statistical distributed rate limiting in an Application Programming Interfaces (API) Gateway cluster. An API Gateway cluster may comprise a plurality of gateway nodes and manage API traffic to ensure proper function and protect the health of an API. Each gateway node may use a distributed rate limiting algorithm based on the physics formulas for average velocity, average acceleration and distance based on time determine the total number of API requests accepted by all the nodes in the cluster. Implementation of statistical distributed rate limiting allows for accurate estimations of the total requests accepted by cluster without requiring each node to share its status with the other nodes in the cluster upon receiving each API request. This approach allows for minimum computational overhead while prioritizing the health of the API.
1. A method for distributed rate limiting of application programming interface (API) requests in an API environment over a period of time, comprising:
calculating, upon receipt of an API request by a first active gateway node within a gateway cluster having a plurality of active gateway nodes, a first velocity and a first acceleration rate of a first subset of the API requests distributed to the first active gateway node;
calculating, by the first active gateway node, a total distance for the plurality of active gateway nodes in the gateway cluster by at least summing a first distance for the first active gateway node based on the first velocity and the first acceleration rate with a second distance based on a second velocity and a second acceleration rate of a second subset of the API requests distributed to at least one remaining active gateway node in the gateway cluster; and;
allowing or denying, by the first active gateway node, the API request to be processed based on determination comparison of the total distance to a predetermined threshold distance; and
transmitting, to the at least one of the remaining active gateway nodes in the gateway cluster by the first active gateway node, the first velocity and the first acceleration rate of the first subset of the API requests,
wherein the first velocity and the first acceleration rate of the first subset of the API requests are usable, by the at least one of the remaining active gateway nodes in the gateway cluster, to allow or deny another API request to be processed.
2. The method of claim 1 , wherein the total distance for the gateway cluster is indicative of the total number of API requests processed by the plurality of active gateway nodes in the gateway cluster during a specified time window, and
wherein the predetermined threshold distance is a distributed quota for the gateway cluster determined by a rate limiting policy defining how many API requests can be processed by resources in the API environment during the specified time window.
3. The method of claim 1 , wherein the calculated first velocity is indicative of a rate of the first subset of the API requests received by the first active gateway node over the period of time.
4. The method of claim 1 , wherein the calculated first acceleration rate of the first subset of the API requests is indicative of a change in the first velocity of the first subset of the API requests received by the first active gateway node over the period of time.
5. The method of claim 1 , further comprising:
asynchronously sending the first velocity and the first acceleration rate of the first subset of the API requests distributed to the first active gateway node to the at least one of the remaining active gateway nodes in the gateway cluster,
wherein sending and receiving velocity and acceleration data occurs at a predefined frequency that is shorter than a specified time window for a rate limiting policy of the gateway cluster.
6. The method of claim 1 , wherein the first active gateway node is configured to deny the API request if the total distance for the gateway cluster is greater than the predetermined threshold distance.
7. The method of claim 1 , wherein the gateway cluster is determined to be under stress if the first acceleration rate of the first subset of the API requests distributed to the first active gateway node exceeds a predetermined acceleration threshold.
8. The method of claim 7 , wherein the gateway cluster is configured to deny the API requests for the plurality of active gateway nodes in the gateway cluster when the gateway cluster is determined to be under stress.
9. The method of claim 1 , wherein the gateway cluster comprises a plurality of gateway nodes and protects a set of resources in the API environment, and
wherein a distributed memory is used to share data across the plurality of active gateway nodes in the gateway cluster.
10. A system comprising:
a memory; and
at least one processor coupled to the memory and configured to:
calculate, upon receipt of an application programming interface (API) request by a first active gateway node within a gateway cluster having a plurality of active gateway nodes, a first velocity and a first acceleration rate of a first subset of API requests distributed to the first active gateway node;
calculate, by the first active gateway node, a total distance for the plurality of active gateway nodes in the gateway cluster by at least summing a first distance for the first active gateway node based on the first velocity and the first acceleration rate with a second distance based on a second velocity and a second acceleration rate of a second subset of the API requests distributed to at least one remaining active gateway node in the gateway cluster; and
allow or deny, by the first active gateway node, the API request to be processed based on determination comparison of the total distance to a predetermined threshold distance; and
transmit, to the at least one of the remaining active gateway nodes in the gateway cluster by the first active gateway node, the first velocity and the first acceleration rate of the API requests associated with the first active gateway node,
wherein the first velocity and the first acceleration rate of the first subset of the API requests are usable, by the at least one of the remaining active gateway nodes in the gateway cluster, to allow or deny another API request to be processed.
11. The system of claim 10 , wherein the total distance for the gateway cluster is indicative of the total number of API requests processed by the plurality of active gateway nodes in the gateway cluster during a specified time window, and
wherein the predetermined threshold distance is a distributed quota for the gateway cluster determined by a rate limiting policy defining how many API requests can be processed by resources in an API environment during the specified time window.
12. The system of claim 10 , wherein the calculated first velocity is indicative of a rate of the first subset of the API requests received by the first active gateway node over a period of time, and
wherein the calculated first acceleration rate of the first subset of the API requests is indicative of a change in the first velocity of the first subset of the API requests received by the first active gateway node over the period of time.
13. The system of claim 10 , wherein the first active gateway node is configured to deny the API request if the total distance for the gateway cluster is greater than the predetermined threshold distance.
14. The system of claim 10 , wherein the gateway cluster is configured to deny the API requests for the plurality of active gateway nodes in the gateway cluster if the first acceleration rate of the first subset of the API requests for the first active gateway node exceeds a predetermined acceleration threshold.
15. The system of claim 10 , wherein the at least one processor is further configured to:
asynchronously send the first velocity and the first acceleration rate of the first subset of the API requests distributed to the first active gateway node to the at least one of the remaining active gateway nodes in the gateway cluster,
wherein sending and receiving velocity and acceleration data occurs at a predefined frequency that is shorter than a specified time window for a rate limiting policy of the gateway cluster, and
wherein a distributed memory is used to share data across a plurality of gateway nodes in the gateway cluster.
16. A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
calculating, upon receipt of an application programming interface (API) request by a first active gateway node within a gateway cluster having a plurality of active gateway nodes, a first velocity and a first acceleration rate of a first subset of API requests distributed to the first active gateway node;
calculating, by the first active gateway node, a total distance for the plurality of active gateway nodes in the gateway cluster by at least summing a first distance for the first active gateway node based on the first velocity and the first acceleration rate with a second distance based on a second velocity and a second acceleration rate of a second subset of the API requests distributed to at least one remaining active gateway node in the gateway cluster; and
allowing or denying, by the first active gateway node, the API request to be processed based on determination comparison of the total distance to a predetermined threshold distance; and
transmitting, to the at least one of the remaining active gateway nodes in the gateway cluster by the first active gateway node, the first velocity and the first acceleration rate of the API requests associated with the first active gateway node,
wherein the first velocity and the first acceleration rate of the first subset of the API requests are usable, by the at least one of the remaining active gateway nodes in the gateway cluster, to allow or deny another API request to be processed.
17. The non-transitory computer-readable device of claim 16 , wherein:
the total distance for the gateway cluster is indicative of the total number of API requests processed by the plurality of active gateway nodes in the gateway cluster during a specified time window,
the predetermined threshold distance is a distributed quota for the gateway cluster determined by a rate limiting policy defining how many API requests can be processed by resources in an API environment during the specified time window, and
the first active gateway node is configured to deny the API request if the total distance for the gateway cluster is greater than the predetermined threshold distance.
18. The non-transitory computer-readable device of claim 16 , wherein the calculated first velocity is indicative of a rate of API requests received by the first active gateway node during a period of time, and
wherein the calculated first acceleration rate of the first subset of the API requests is indicative of a change in the first velocity of the first subset of the API requests received by the first active gateway node over the period of time.
19. The non-transitory computer-readable device of claim 16 , the operations further comprising:
asynchronously sending the first velocity and the first acceleration rate of the first subset of the API requests distributed to the first active gateway node to the at least one of the remaining active gateway nodes in the gateway cluster,
wherein sending and receiving velocity and acceleration data occurs at a predefined frequency that is shorter than a specified time window for a rate limiting policy of the gateway cluster, and
wherein a distributed memory is used to share data across a plurality of gateway nodes in the gateway cluster.