Dynamic buffer management in multi-client token flow control routers
Systems, apparatuses, and methods for dynamic buffer management in multi-client token flow control routers are disclosed. A system includes at least one or more processing units, a memory, and a communication fabric with a plurality of routers coupled to the processing unit(s) and the memory. A router servicing multiple active clients allocates a first number of tokens to each active client. The first number of tokens is less than a second number of tokens needed to saturate the bandwidth of each client to the router. The router also allocates a third number of tokens to a free pool, with tokens from the free pool being dynamically allocated to different clients. The third number of tokens is equal to the difference between the second number of tokens and the first number of tokens. An advantage of this approach is reducing the amount of buffer space needed at the router.
1. A communication fabric having a maximum bandwidth, the maximum bandwidth corresponding to a maximum number of tokens, the communication fabric comprising:
a router comprising circuitry configured to allocate tokens to a plurality of clients according to a token allocation scheme, wherein each of the plurality of clients is configured to access a shared resource via the router, wherein the token allocation scheme comprises:
statically allocating a first number of tokens to at least one client that is not active of the plurality of clients, the first number of tokens being:
at least one token; and
less than the maximum number of tokens; and
dynamically allocating to each active client of the plurality of clients a second number of tokens from a free pool, the second number of tokens from the free pool is less than or equal to a difference between the maximum number of tokens and the first number of tokens;
deallocate one or more tokens from a given client while allowing the given client to maintain at least one token, based at least in part on a determination that the given client is no longer active; and
wherein a total buffering requirement in the router corresponding to a total number of packets for the plurality of clients is equal to a sum of the maximum number of tokens and a product of the first number of tokens and a total number of clients of the plurality of clients less one.
2. The communication fabric as recited in claim 1 , wherein statically allocating, at a beginning of a token allocation round, the first number of tokens is performed per unit time.
3. The communication fabric as recited in claim 1 , wherein a total number of tokens available to allocate to the plurality of clients is less than a product of a number of the plurality of clients and the maximum number of tokens corresponding to the maximum bandwidth.
4. The communication fabric as recited in claim 1 , wherein dynamically allocating, during a token allocation round, the second number of tokens is performed over a unit time.
5. The communication fabric as recited in claim 1 , wherein the router is further configured to, based at least in part on a determination that two or more active clients are contending for the shared resource, dynamically allocate the second number of tokens to each active client of the plurality of clients based on one or more arbitration weights.
6. The communication fabric as recited in claim 1 , wherein the router is configured to deallocate the one or more tokens from the given client in further response to a determination that the given client had been previously active.
7. A method comprising:
statically allocating, by a router comprising circuitry, a first number of tokens to at least one client that is not active of a plurality of clients, wherein each of the plurality of clients is configured to access a shared resource via the router, the first number of tokens being:
at least one token; and
less than a maximum number of tokens corresponding to a maximum bandwidth of a communication fabric; and
dynamically allocating, by the router, to each active client of the plurality of clients a second number of tokens from a free pool, the second number of tokens being less than or equal to a difference between the maximum number of tokens and the first number of tokens;
deallocating one or more tokens from a given client while allowing the given client to maintain at least one token, based at least in part on a determination that the given client is no longer active; and
wherein a total buffering requirement in the router corresponding to a total number of packets for the plurality of clients is equal to a sum of the maximum number of tokens and a product of the first number of tokens and a total number of clients of the plurality of clients less one.
8. The method as recited in claim 7 , wherein statically allocating, at a beginning of a token allocation round, the first number of tokens is performed per unit time.
9. The method as recited in claim 7 , wherein a total number of tokens available to allocate to the plurality of clients is less than a product of a number of the plurality of clients and the maximum number of tokens corresponding to the maximum bandwidth.
10. The method as recited in claim 7 , wherein dynamically allocating, during a token allocation round, the second number of tokens is performed over a unit time.
11. The method as recited in claim 7 , further comprising dynamically allocating the second number of tokens based on an activity level of the active client.
12. The method as recited in claim 7 , further comprising dynamically allocating the second number of tokens to each active client of the plurality of clients based on one or more arbitration weights, responsive to determining that two or more active clients are contending for the shared resource.
13. An apparatus comprising:
a free pool of tokens;
a plurality of buffers; and
a router comprising circuitry configured to:
statically allocate a first number of tokens to at least one client that is not active of a plurality of clients, wherein each of the plurality of clients is configured to access a shared resource via the router, the first number of tokens being:
at least one token; and
less than a maximum number of tokens corresponding to a maximum bandwidth of a communication fabric; and
dynamically allocate, to each active client of the plurality of clients, one or more tokens from the free pool, the one or more tokens representing a difference between the maximum number of tokens and the first number of tokens currently allocated to the active client; and
deallocate one or more tokens from a given client while allowing the given client to maintain at least one token, based at least in part on a determination that the given client is no longer active; and
update a number of tokens in the free pool such that a number of tokens available for allocation corresponds to available buffer space in the plurality of buffers;
wherein a total buffering requirement in the router corresponding to a total number of packets for the plurality of clients is equal to a sum of the maximum number of tokens and a product of the first number of tokens and a total number of clients of the plurality of clients less one.
14. The apparatus as recited in claim 13 , wherein statically allocating, at a beginning of a token allocation round, the first number of tokens is performed per unit time.
15. The apparatus as recited in claim 13 , wherein a total number of tokens available to allocate to the plurality of clients is less than a product of a number of the plurality of clients and the maximum number of tokens corresponding to the maximum bandwidth.
16. The apparatus as recited in claim 13 , wherein dynamically allocating, during a token allocation round, a second number of tokens is performed over a unit time.
17. The apparatus as recited in claim 13 , wherein the apparatus is further configured to dynamically allocate, during a token allocation round, a second number of tokens based on an activity level of the active client.