Granted Patent B2

US 12,301,472 · App. 17/407,910 · Granted May 13, 2025

Inventors: Raghuveer Murari Bhandarkar (Bangalore, IN); Ravijeet Ranjit Kumar (Bangalore, IN)

Assignee: Oracle International Corporation

H04L47/6225H04L43/16H04L47/562H04L47/6255

Abstract

Claims (87)

1. A computer-implemented method, comprising:

receiving, at a throttler engine of a distributed computing system, a plurality of requests to use a software component, each request of the plurality of requests associated with a request time;

aggregating, by the throttler engine, a subset of the plurality of requests based at least in part on a time window, wherein the request time of each of the requests in the subset corresponds to the time window, and wherein the subset is aggregated into a queue;

determining, based at least in part on the subset of the plurality of requests, a received request rate;

identifying a request rate limit associated with the software component, the request rate limit corresponding to the time window;

tuning a plurality of weights;

computing, based at least in part on the received request rate, the request rate limit and the plurality of weights, a plurality of correction values;

determining a throttled request rate based at least in part on the plurality of correction values;

identifying one or more throttled requests from the subset, the one or more throttled requests corresponding to the throttled request rate; and

providing the one or more throttled requests to the software component,

wherein tuning the plurality of weights comprises:

detecting (i) a deviation between a predicted request rate and the received request rate and (ii) a previous deviation between the predicted request rate and a previous received request rate, wherein a drift corresponds to a difference between the deviation and the previous deviation; and

responsive to detecting the drift exceeding a threshold, re-tuning the plurality of weights by simulating, by the throttler engine, the throttled request rate using a new predicted request rate.

2. The computer-implemented method of claim 1 , further comprising:

aggregating a second subset of the plurality of requests based at least in part on a second time window, the request times of each of the requests in the second subset corresponding to the second time window, the second subset aggregated into the queue;

determining, based at least in part on the second subset, a second received request rate corresponding to the second time window;

updating, based at least in part on the second received request rate, the plurality of correction values;

updating the throttled request rate based at least in part on the updated plurality of correction values;

identifying additional throttled requests from the second subset, the additional throttled requests corresponding to the updated throttled request rate; and

providing the additional throttled requests to the software component.

3. The computer-implemented method of claim 2 , wherein computing the plurality of correction values comprises:

computing a proportional correction value based at least in part on a first difference between the received request rate and the request rate limit, the proportional correction value comprising a proportional weight;

computing an integral correction value based at least in part on the first difference and a second difference between the second received request rate and the request rate limit, the integral correction value comprising an integral weight; and

computing a smoothed correction value based at least in part on the first difference and the second difference, the smoothed correction value comprising a smoothed weight.

4. The computer-implemented method of claim 1 , further comprising:

determining the new predicted request rate based at least in part on the previous deviation.

5. The computer-implemented method of claim 1 , wherein the plurality of weights tuned using hyperparameter tuning comprises predetermined values.

6. The computer-implemented method of claim 1 , wherein the queue comprises a first-in first-out queue.

7. The computer-implemented method of claim 1 , wherein the request rate limit comprises a static request rate limit.

8. The computer-implemented method of claim 1 , wherein tuning the plurality of weights further comprises using hyperparameter tuning and performing a grid search.

9. The computer-implemented method of claim 1 , wherein the predicted request rate is based at least in part on historical request traffic to the software component.

10. The computer-implemented method of claim 1 , wherein the previous received request rate is determined based at least in part on previous requests received during a previous time window preceding the time window.

11. A distributed computing system comprising:

one or more processors; and

one or more memories storing computer-executable instructions that, when executed with the one or more processors, cause the distributed computing system to:

receive, at a throttler engine of the distributed computing system, a plurality of requests to use a software component, each request of the plurality of requests associated with a request time;

aggregate, by the throttler engine, a subset of the plurality of requests based at least in part on a time window, wherein the request time of each of the requests in the subset corresponds to the time window, and wherein the subset is aggregated into a queue;

determine, based at least in part on the subset of the plurality of requests, a received request rate;

identify a request rate limit associated with the software component, the request rate limit corresponding to the time window;

tune a plurality of weights;

compute, based at least in part on the received request rate, the request rate limit and the plurality of weights, a plurality of correction values;

determine a throttled request rate based at least in part on the plurality of correction values;

identify one or more throttled requests from the subset, wherein the one or more throttled requests correspond to the throttled request rate; and

provide the one or more throttled requests to the software component,

wherein tuning the plurality of weights comprises:

detecting (i) a deviation between a predicted request rate and the received request rate and (ii) a previous deviation between the predicted request rate and a previous received request rate, wherein a drift corresponds to a difference between the deviation and the previous deviation; and

responsive to detecting the drift exceeding a threshold, re-tuning the plurality of weights by simulating, by the throttler engine, the throttled request rate using a new predicted request rate.

12. The distributed computing system of claim 11 , wherein executing the computer-executable instructions further causes the distributed computing system to:

aggregate a second subset of the plurality of requests based at least in part on a second time window, the request times of each of the requests in the second subset corresponding to the second time window, the second subset aggregated into the queue;

determine, based at least in part on the second subset, a second received request rate corresponding to the second time window;

update, based at least in part on the second received request rate, the plurality of correction values;

update the throttled request rate based at least in part on the updated plurality of correction values;

identify additional throttled requests from the second subset, the additional throttled requests corresponding to the updated throttled request rate; and

provide the additional throttled requests to the software component.

13. The distributed computing system of claim 12 , wherein computing the plurality of correction values comprises:

computing a proportional correction value based at least in part on a first difference between the received request rate and the request rate limit, the proportional correction value comprising a proportional weight;

computing an integral correction value based at least in part on the first difference and a second difference between the second received request rate and the request rate limit, the integral correction value comprising an integral weight; and

computing a smoothed correction value based at least in part on the first difference and the second difference, the smoothed correction value comprising a smoothed weight.

14. The distributed computing system of claim 11 , wherein executing the computer-executable instructions further causes the distributed computing system to:

determine the new predicted request rate based at least in part on the previous deviation.

15. The distributed computing system of claim 11 , wherein tuning the plurality of weights further comprises using hyperparameter tuning and performing a grid search.

16. The distributed computing system of claim 11 , wherein the predicted request rate is based at least in part on historical request traffic to the software component.

17. The distributed computing system of claim 11 , wherein the previous received request rate is determined based at least in part on previous requests received during a previous time window preceding the time window.

18. A non-transitory computer readable medium storing computer-executable instructions that, when executed by one or more processors, cause a distributed computing system to at least:

receive, at a throttler engine of the distributed computing system, a plurality of requests to use a software component, each request of the plurality of requests associated with a request time;

aggregate, by the throttler engine, a subset of the plurality of requests based at least in part on a time window, wherein the request time of each of the requests in the subset corresponds to the time window, and wherein the subset is aggregated into a queue;

determine, based at least in part on the subset of the plurality of requests, a received request rate;

identify a request rate limit associated with the software component, the request rate limit corresponding to the time window;

tune a plurality of weights;

compute, based at least in part on the received request rate, the request rate limit, and the plurality of weights, a plurality of correction values;

determine a throttled request rate based at least in part on the plurality of correction values;

identify one or more throttled requests from the subset, wherein the one or more throttled requests correspond to the throttled request rate; and

provide the one or more throttled requests to the software component,

wherein tuning the plurality of weights comprises:

detecting (i) a deviation between a predicted request rate and the received request rate and (ii) a previous deviation between the predicted request rate and a previous received request rate, wherein a drift corresponds to a difference between the deviation and the previous deviation; and

responsive to detecting the drift exceeding a threshold, re-tuning the plurality of weights by simulating, by the throttler engine, the throttled request rate using a new predicted request rate.

19. The non-transitory computer readable medium of claim 18 , wherein executing the computer-executable instructions further causes the distributed computing system to:

aggregate a second subset of the plurality of requests based at least in part on a second time window, the request times of each of the requests in the second subset corresponding to the second time window, the second subset aggregated into the queue;

determine, based at least in part on the second subset, a second received request rate corresponding to the second time window;

update, based at least in part on the second received request rate, the plurality of correction values;

update the throttled request rate based at least in part on the updated plurality of correction values;

identify additional throttled requests from the second subset, the additional throttled requests corresponding to the updated throttled request rate; and

provide the additional throttled requests to the software component.

20. The non-transitory computer readable medium of claim 19 , wherein computing the plurality of correction values comprises:

computing a proportional correction value based at least in part on a first difference between the received request rate and the request rate limit, the proportional correction value comprising a proportional weight;

computing an integral correction value based at least in part on the first difference and a second difference between the second received request rate and the request rate limit, the integral correction value comprising an integral weight; and

computing a smoothed correction value based at least in part on the first difference and the second difference, the smoothed correction value comprising a smoothed weight.

Assignments (1)

Continuity (1)

References Cited (17)