Dynamic buffer management in multi-client token flow control routers
Systems, apparatuses, and methods for dynamic buffer management in multi-client token flow control routers are disclosed. A system includes at least one or more processing units, a memory, and a communication fabric with a plurality of routers coupled to the processing unit(s) and the memory. A router servicing multiple active clients allocates a first number of tokens to each active client. The first number of tokens is less than a second number of tokens needed to saturate the bandwidth of each client to the router. The router also allocates a third number of tokens to a free pool, with tokens from the free pool being dynamically allocated to different clients. The third number of tokens is equal to the difference between the second number of tokens and the first number of tokens. An advantage of this approach is reducing the amount of buffer space needed at the router.
1. A system, comprising:
a plurality of processing units;
a memory; and
a communication fabric coupled to the one or more processing units and the memory, wherein the communication fabric comprises a router configured to:
receive packets from a first client, wherein a first number of tokens per unit time are required to saturate bandwidth from the first client to the router;
statically allocate a second number of tokens per unit time to the first client, wherein the second number of tokens is less than the first number of tokens;
maintain a free pool of tokens for dynamic allocation to a plurality of clients, wherein the plurality of clients includes the first client;
receive packets from the plurality of clients, wherein a corresponding number of tokens per unit time are required to saturate bandwidth from each client of the plurality of clients to the router;
statically allocate a number of tokens per unit time to each client that is less than the corresponding number of tokens per unit time required to saturate bandwidth for the respective client;
determine how many clients targeting a given destination are active; and
calculate a token threshold per client based on the number of active clients, wherein the token threshold for the first client is between the first number and the second number.
2. The system as recited in claim 1 , wherein the router is further configured to:
define the token threshold per client as a maximum number of tokens that can be allocated by the router to a given client per unit time; and
wherein the token threshold is further calculated based on an arbitration weight assigned to each client.
3. The system as recited in claim 1 , wherein:
the plurality of clients target a common destination; and
a number of tokens in the free pool is equal to the first number subtracted by the second number.
4. The system as recited in claim 1 , wherein the first token flow control router is configured to dynamically allocate tokens from the free pool to clients based on a priority level assigned to each client.
5. The system as recited in claim 1 , wherein the first token flow control router is configured to dynamically allocate tokens from the free pool to clients based on a bandwidth level requested by each client.
6. The system as recited in claim 1 , wherein the router comprises buffers sized to store the second number of packets from each client of the plurality of clients and a number of packets equal to a number of tokens in the free pool.
7. A method, comprising:
receiving packets from a first client, wherein a first number of tokens per unit time are required to saturate bandwidth from the first client to the router;
statically allocating a second number of tokens per unit time to the first client, wherein the second number of tokens is less than the first number of tokens;
maintaining a free pool of tokens for dynamic allocation to a plurality of clients, wherein the plurality of clients includes the first client;
receiving packets from the plurality of clients, wherein a corresponding number of tokens per unit time are required to saturate bandwidth from each client of the plurality of clients to the router;
statically allocating a number of tokens per unit time to each client that is less than the corresponding number of tokens per unit time required to saturate bandwidth for the respective client;
determining how many clients targeting a given destination are active; and
calculating a token threshold per client based on the number of active clients, wherein the token threshold for the first client is between the first number and the second number.
8. The method as recited in claim 7 , further comprising:
defining the token threshold per client as a maximum number of tokens that can be allocated by the router to a given client per unit time; and
wherein the token threshold is further calculated based on an arbitration weight assigned to each client.
9. The method as recited in claim 7 , wherein:
the plurality of clients target a common destination; and
a number of tokens in the free pool is equal to the first number subtracted by the second number.
10. The method as recited in claim 7 , further comprising dynamically allocating tokens from the free pool to clients based on a priority level assigned to each client.
11. The method as recited in claim 7 , further comprising dynamically allocating tokens from the free pool to clients based on a bandwidth level requested by each client.
12. The method as recited in claim 7 , further comprising maintaining buffers sized to store the second number of packets from each client of the plurality of clients and a number of packets equal to a number of tokens in the free pool.
13. An apparatus, comprising:
a plurality of processing units; and
a router;
wherein the router is configured to:
receive packets from a first client, wherein a first number of tokens per unit time are required to saturate bandwidth from the first client to the router;
statically allocate a second number of tokens per unit time to the first client, wherein the second number of tokens is less than the first number of tokens;
maintain a free pool of tokens for dynamic allocation to a plurality of clients, wherein the plurality of clients includes the first client;
receive packets from the plurality of clients, wherein a corresponding number of tokens per unit time are required to saturate bandwidth from each client of the plurality of clients to the router;
statically allocate a number of tokens per unit time to each client that is less than the corresponding number of tokens per unit time required to saturate bandwidth for the respective client;
determine how many clients targeting a given destination are active; and
calculate a token threshold per client based on the number of active clients, wherein the token threshold for the first client is between the first number and the second number.
14. The apparatus as recited in claim 13 , wherein the router is further configured to:
define the token threshold per client as a maximum number of tokens that can be allocated by the router to a given client per unit time; and
wherein the token threshold is further calculated based on an arbitration weight assigned to each client.
15. The apparatus as recited in claim 13 , wherein:
the plurality of clients target a common destination; and
a number of tokens in the free pool is equal to the first number subtracted by the second number.
16. The apparatus as recited in claim 13 , wherein the token flow control router is configured to dynamically allocate tokens from the free pool to clients based on a bandwidth level requested by each client.
17. The apparatus as recited in claim 13 , wherein the router comprises buffers sized to store a second number of packets from each client of the plurality of clients and a number of packets equal to a number of tokens in the free pool.