IP Library Granted Patent US 12,238,009
Granted Patent B1
US 12,238,009 · App. 18/377,097 · Granted Feb 25, 2025

Distributed rate limiting

Inventors: Peter John Hill (Seattle, WA); Michal Aleksander Karczmarek (Newton, MA); Eugen Enache (Bucharest, RO)
Assignee: Oracle International Corporation
H04L47/2433H04L43/0876H04L47/215H04L67/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,238,009
App. No.
18/377,097
Granted
Feb 25, 2025
Kind
B1
Abstract

Techniques for performing distributed rate limiting in networks in a cloud environment are described for determining an amount of network bandwidth available to be processed by flow control nodes within a cloud network for a first time period, determining a bandwidth allocation for traffic classes for the first time period, determining, a portion of the bandwidth allocation for the flow control nodes, providing, data to the flow control nodes, where the data indicates the portion of the bandwidth allocation for the traffic classes, and receiving, second data that indicates an amount of network traffic routed during the first time period by individual ones of the flow control nodes.

Claims (60)

1. A method, comprising:

determining, via one or more processors, an amount of network bandwidth available to be processed by flow control nodes within a cloud network for a first time period;

determining, via the one or more processors, a bandwidth allocation for traffic classes for the first time period;

determining, via the one or more processors, a portion of the bandwidth allocation for the flow control nodes;

providing, via the one or more processors, data to the flow control nodes, where the data indicates the portion of the bandwidth allocation for the traffic classes; and

receiving, via the one or more processors, second data that indicates an amount of network traffic routed during the first time period by individual ones of the flow control nodes.

2. The method of claim 1 , wherein determining the bandwidth allocation for the traffic classes includes determining a total number of tokens to be generated within individual ones of the flow control nodes during the first time period, where the total number of tokens is based, at least in part, on an amount of network bandwidth that the individual ones of the flow control nodes can process during the first time period.

3. The method of claim 1 , wherein providing the flow control nodes with the data that indicates the portion of the bandwidth allocation for the traffic classes includes a total number of tokens to be generated within the flow control node for the first time period, as well as a minimum guaranteed token allocation and a maximum allowable token allocation for individual ones of the traffic classes during the first time period.

4. The method of claim 1 , further comprising receiving, via the one or more processors, from individual ones of the flow control nodes token use data for individual ones of the traffic classes during the first time period.

5. The method of claim 1 , wherein, for individual ones of the flow control nodes, adjusting one or more of a minimum guaranteed token allocation or a maximum allowable token allocation for one or more of individual ones of the traffic classes, or one or more individual flow control nodes, where the adjusting is based at least in part on an analysis of the second data.

6. The method of claim 1 , where providing the data to the flow control nodes, includes sending to the flow control nodes a total number of tokens to be generated within the first time period, as well as a minimum guaranteed token allocation and a maximum allowable token allocation for each of the traffic classes during the first time period.

7. The method of claim 1 , further comprising:

determining that a first flow control node of the flow control nodes utilized a first number of tokens for a first traffic class during the first time period;

determining that a second flow control node utilized a second number of tokens for the first traffic class during the first time period;

determining that the first number of tokens is different than the second number of tokens; and

adjusting one or more of a minimum guaranteed token allocation or a maximum allowable token allocation for one or more of the first flow control node or the second control node.

8. The method of claim 1 , further comprising:

determining that a first flow control node of the flow control nodes utilized a first number of tokens for a first traffic class during the first time period;

determining that a second flow control node utilized a second number of tokens for the first traffic class during the first time period;

determining that the first number of tokens is different than the second number of tokens; and

adjusting one or more of a minimum guaranteed token allocation or a maximum allowable token allocation for one or more of the first traffic class or one or more other traffic classes.

9. A system comprising:

a management node within a cloud network; and

a flow control node within the cloud network, wherein the flow control node is configured to:

receive data from the management node, wherein the data indicates one or more bandwidth allocations for traffic classes available for the flow control node to process during a first time period;

receive network data to be routed to a destination within the cloud network;

identify a traffic class associated with the network data;

determine a number of tokens associated with the traffic class, wherein an individual token represents a predetermined amount of network data that can be routed by the flow control node;

perform one or more operations based, at least in part, on the number of tokens associated with the traffic class, where the one or more operations include one or more of routing the data to the destination, or dropping the network data and providing one or more errors;

provide second data to the management node, where the second data indicates a portion of the bandwidth allocation for the traffic classes used by the flow control node to route network data during the time period; and

receive third data from the management node, wherein the third data indicates one or more second bandwidth allocations for the traffic classes available for the flow control node to process during a second time period.

10. The system of claim 9 , where the flow control node is further configured to:

generate a total number of tokens for the flow control node; and

distribute the total number of tokens to class-specific buckets according to a minimum guaranteed token allocation and a maximum allowable token allocation for each of the traffic classes.

11. The system of claim 9 , wherein the flow control node is further configured to:

receive updated data from the management node, wherein the updated data indicates one or more updated bandwidth allocations for the traffic classes available for the flow control node to process during a second time period; and

update a minimum guaranteed token allocation and a maximum allowable token allocation for individual ones of class-specific buckets corresponding to the traffic classes.

12. The system of claim 9 , wherein the flow control node is further configured to:

determine that the number of tokens held within s class-specific bucket associated with the traffic class is insufficient to route the network data; and

wherein performing the one or more operations includes dropping the network data.

13. The system of claim 9 , wherein the flow control node is further configured to:

determine that the number of tokens held within a class-specific bucket for the traffic class is sufficient to route the network data; and

wherein performing the one or more operations includes routing the network data to the destination.

14. The system of claim 13 , wherein the flow control node is further configured to remove a number of tokens from the class-specific bucket, wherein the number of tokens corresponds to an amount of the network data routed.

15. The system of claim 9 , wherein the flow control node is further configured to:

determine that the number of tokens held within a class-specific bucket associated with the traffic class is insufficient to route the network data;

determine that the flow control node has exceeded a maximum token allocation for the traffic class for the time period; and

wherein performing the one or more operations includes re-allocating a first portion of tokens from at least one of the other traffic classes to the traffic class associated with the network data and routing the network data to the destination.

16. The system of claim 9 , wherein the flow control node is further configured to:

determine that all of the tokens are removed from a class-specific bucket before an expiration of the first time period; and

re-allocating tokens from one or more other class-specific buckets to the class-specific bucket before the expiration of the first time period.

17. A non-transitory computer-readable medium storing a set of instructions, the set of instructions when executed by one or more processors cause processing to be performed comprising:

determining an amount of network bandwidth available to be processed by flow control nodes within a cloud network for a first time period;

determining a bandwidth allocation for traffic classes for the first time period;

determining a portion of the bandwidth allocation for the flow control nodes;

providing data to the flow control nodes, where the data indicates the portion of the bandwidth allocation for the traffic classes; and

receiving second data that indicates an amount of network traffic routed during the first time period by individual ones of the flow control nodes.

18. The non-transitory computer-readable medium of claim 17 , wherein determining the bandwidth allocation for the traffic classes includes determining a total number of tokens to be generated within individual ones of the flow control nodes during the first time period, where the total number of tokens is based, at least in part, on an amount of network bandwidth that the flow control node can process during the first time period.

19. The non-transitory computer-readable medium of claim 17 , wherein providing the flow control nodes with the data that indicates the portion of the bandwidth allocation for the traffic classes includes a total number of tokens to be generated within the flow control node for the first time period, as well as a minimum guaranteed token allocation and a maximum allowable token allocation for individual ones of the traffic classes during the first time period.

20. The non-transitory computer-readable medium of claim 17 , wherein the processing to be performed further comprises obtaining, from individual ones of the flow control nodes, token utilization information for individual ones of the traffic classes during the first time period.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2023
From: HILL, PETER JOHN; KARCZMAREK, MICHAL ALEKSANDER; ENACHE, EUGEN
To: ORACLE INTERNATIONAL CORPORATION
Reel/Frame 065138/0326 →
References Cited (33)
US 7006437B2 · Ogier · 2006 [cited by examiner]
US 7142558B1 · Pack · 2006 [cited by examiner]
US 7680897B1 · Carter · 2010 [cited by examiner]
US 7684333B1 · Dasylva · 2010 [cited by examiner]
US 7738375B1 · Vinokour · 2010 [cited by examiner]
US 8027345B2 · Ayyagari · 2011 [cited by examiner]
US 8204069B2 · Jones · 2012 [cited by examiner]
US 8392571B2 · Schlansker · 2013 [cited by examiner]
US 8681614B1 · McCanne · 2014 [cited by examiner]
US 9042355B2 · Thomasson · 2015 [cited by examiner]
US 9326051B2 · Skubic · 2016 [cited by examiner]
US 10177993B2 · Chitti · 2019 [cited by examiner]
US 10764206B2 · Boss · 2020 [cited by examiner]
US 10785053B2 · Kerpez · 2020 [cited by examiner]
US 11381452B2 · Huang · 2022 [cited by examiner]
US 11431645B2 · Zhou · 2022 [cited by examiner]
US 11522887B2 · Dunn · 2022 [cited by examiner]
US 11531325B2 · Nixon · 2022 [cited by examiner]
US 11726460B2 · Nixon · 2023 [cited by examiner]
US 11824650B2 · Nixon · 2023 [cited by examiner]
US 11824934B2 · Nixon · 2023 [cited by examiner]
US 20140195688A1 · Archer et al. · 2014 [cited by applicant]
US 20200120036A1 · Zhou · 2020 [cited by examiner]
US 20220210091A1 · Wu · 2022 [cited by applicant]
US 20240250917A1 · Milano · 2024 [cited by examiner]
CN 101060471A · 2007 [cited by applicant]
CN 108848038A · 2018 [cited by applicant]
CN 115632990A · 2023 [cited by applicant]
“Rate Limiting Pattern”, Available online at https://learn.microsoft.com/en-us/azure/architecture/patterns/rate-limiting-pattern, 2023, 10 pages. [cited by applicant]
“Rate-Limiting Strategies and Techniques”, Google Cloud, Available online at: https://cloud.google.com/solutions/rate-limiting-strategies-techniques, Mar. 24, 2023, 13 pages. [cited by applicant]
Raghavan , et al., “Cloud Control with Distributed Rate Limiting”, SIGCOMM '07: Proceedings of the 2007 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, Available onlin… [cited by applicant]
Wang , et al., “An Application-Level QoS Control Method Based on Local Bandwidth Scheduling”, Journal of Electrical and Computer Engineering, vol. 2018, Article ID 4576245, Jun. 2018, pp. 1-10. [cited by applicant]
Zhang , et al., “TFC: Token Flow Control in Data Center Networks”, EuroSys '16: Proceedings of the Eleventh European Conference on Computer Systems, Available online at http://nns.cs.tsinghua.edu.cn/paper/eurosys16_jz.p… [cited by applicant]