Data flow control in distributed computing systems
Datacenters or other large-scale distributed computing systems can provide computing resources such as processing power and data storage as computing services accessible to tenants via a computer network. A tenant, such as a corporation, school, or organization, can have multiple users or groups of users with corresponding websites. To facilitate ready access, data relevant to a user, group, or website of a tenant can be stored in a dedicated network location sometimes referred to as a shard. A shard can be a physical and/or logical storage location that contains emails, chats, instant messages, documents, photos, videos, or other types of content items with which the user, group, or website can interact.
1. A computer implemented method, comprising:
receiving, at a token issuer component, a token request that corresponds to a token requestor and requests a token representing permission to record a file interaction in an index;
identifying a particular category of file interaction for which the token is requested;
accessing a portion of a token bucket structure of token buckets based on the particular category of file interactions, the token bucket structure including:
a first token bucket corresponding to a first type of file interaction belonging to a first category of file interactions, the first token bucket having a first number of available tokens, and
a second token bucket corresponding to a second type of file interaction belonging to a second category of file interactions, the second type of file interaction being different than the first type of file interaction, and the second token bucket having a second number of available tokens; and
based on determining that there are enough available tokens, in one of the first token bucket or the second token bucket corresponding to the particular category of file interactions, to service the token request,
providing the token to the token requestor.
2. The computer implemented method of claim 1 , and further comprising:
receiving a second token request that requests a second token;
identifying a second particular category of file interaction for which the second token is requested;
based on determining that there are insufficient available tokens, in one of the first token bucket or the second token bucket corresponding to the second particular category of file interactions, to service the second token request, generating an output denying the second token request.
3. The computer implemented method of claim 1 and further comprising:
detecting refill criteria indicating that a refill operation is to be performed; and
distributing a number of tokens to each token bucket in the token bucket structure based on an overall number of tokens allocated to the token bucket structure.
4. The computer implemented method of claim 3 wherein distributing a number of tokens to each token bucket in the token bucket structure comprises:
distributing a number of tokens to each token bucket in the token bucket structure based on a number of token requests received for each token bucket in the token bucket structure during a previous time interval.
5. The computer implemented method of claim 1 , and further comprising:
for each given token bucket of one or more token buckets in the token bucket structure,
calculating a proportion of an overall number of token requests received for the given token bucket during a previous time interval; and
distributing a particular number of tokens to the given token bucket in the token bucket structure based on the proportion of the overall number of token requests received for the given token bucket.
6. The computer implemented method of claim 1 , and further comprising:
for each given token bucket of one or more token buckets in the token bucket structure,
identifying a proportion of an overall number of tokens to be distributed to the given token bucket; and
distributing the proportion of the overall number of tokens to the given token bucket.
7. The computer implemented method of claim 6 wherein identifying a proportion of the overall number of tokens to be distributed to the given token bucket comprises:
identifying an overall number of token requests received by the token buckets during a previous time interval; and
for each respective token bucket in the token bucket structure:
identifying a particular number of token requests received for the respective token bucket during the previous time interval;
calculating a particular proportion of received token requests for the given token bucket as the proportion of the overall number of token requests comprised by the particular number of token requests; and
identifying the proportion of the overall number of tokens to be distributed to the given token bucket based on the particular proportion of received token requests.
8. The computer implemented method of claim 1 wherein the token bucket structure includes a hierarchical token bucket structure having a parent token bucket and a child token bucket and wherein accessing a portion of the token bucket structure comprises:
accessing a portion of the hierarchical token bucket structure.
9. The computer implemented method of claim 8 wherein accessing a portion of the hierarchical token bucket structure comprises:
accessing the child token bucket that comprises a first category token bucket in the hierarchical token bucket structure, corresponding to the first category of file interactions, to determine whether there are enough tokens available to service the token request; and
based on determining there are not enough tokens available to service the token request, accessing the parent token bucket, that is a parent of the first category token bucket, to determine whether there are enough tokens available to service the token request.
10. The computer implemented method of claim 9 wherein the parent token bucket comprises a shared token bucket that is shared by a plurality of child token buckets.
11. A computer implemented method, comprising:
receiving a token request at a token issuer component, the token request requesting a token representing permission to record a file interaction in an index, the file interaction being of a first type of file interaction belonging to a first category of file interactions;
accessing a portion of a token bucket structure of token buckets based on the first category of file interactions to determine whether there are enough tokens available to service the token request, each token bucket in the token bucket structure corresponding to a different category of file interactions and having a number of available tokens;
if there are enough tokens available to service the token request, issuing a number of tokens to serve the token request;
decrementing a number of available tokens in the portion of the token bucket structure based on the number of tokens issued to serve the token request;
identifying refill criteria indicating that tokens are to be added to one or more token buckets in the token bucket structure; and
performing a refill operation comprising:
counting a number of token requests received for each token bucket in the token bucket structure during a previous time interval; and
distributing a number of tokens to the one or more token buckets in the token bucket structure based on the number of token requests received for each token bucket in the token bucket structure during the previous time interval.
12. The computer implemented method of claim 11 wherein performing a refill operation comprises:
for each token bucket, adding a predefined quantity of tokens to the token bucket.
13. The computer implemented method of claim 11 wherein the token bucket structure includes a hierarchical token bucket structure having a parent token bucket and a child token bucket and wherein accessing a portion of the token bucket structure comprises:
accessing a portion of the hierarchical token bucket structure.
14. The computer implemented method of claim 13 wherein the child token bucket comprises a first category token bucket corresponding to the first category of file interactions and wherein accessing a portion of the hierarchical token bucket structure comprises:
accessing the child token bucket to determine whether there are enough tokens available to service the token request; and
if not, accessing the parent token bucket to determine whether there are enough tokens available to service the token request.
15. The computer implemented method of claim 13 wherein the parent token bucket comprises a shared token bucket that is shared by a plurality of child token buckets.
16. A computer system, comprising:
at least one processor; and
a data store storing instructions which, when executed by the at least one processor cause the at least one processor to perform steps, comprising:
receiving, at a token issuer component, a token request that corresponds to a token requestor and requests a token representing permission to record a file interaction in an index;
identifying a particular category of file interaction for which the token is requested;
accessing a portion of a token bucket structure of token buckets based on the particular category of file interactions, the token bucket structure including:
a first token bucket corresponding to a first type of file interaction belonging to a first category of file interactions, the first token bucket having a first number of available tokens, and
a second token bucket corresponding to a second type of file interaction belonging to a second category of file interactions, the second type of file interaction being different than the first type of file interaction, and the second token bucket having a second number of available tokens;
based on determining that there are enough tokens, in one of the first token bucket or the second token bucket corresponding to the particular category of file interactions, available to service the token request, issuing a number of tokens to serve the token request; and
decrementing a number of available tokens in the portion of the token bucket structure based on the number of tokens issued to serve the token request.
17. The computer system of claim 16 wherein the instructions cause the at least one processor to perform steps further comprising:
identifying refill criteria indicating that tokens are to be added to one or more token buckets in the token bucket structure; and
performing a refill operation to add tokens to the one or more token buckets.
18. The computer system of claim 16 wherein the instructions cause the at least one processor to perform steps further comprising:
counting a number of token requests received for each token bucket in the token bucket structure during a previous time interval; and
distributing a number of tokens to one or more token buckets in the token bucket structure based on the number of token requests received for the one or more token buckets during the previous time interval.
19. The computer system of claim 16 wherein the token bucket structure includes a hierarchical token bucket structure having a parent token bucket and a child token bucket and wherein accessing a portion of the token bucket structure comprises:
accessing a portion of the hierarchical token bucket structure.
20. The computer system of claim 19 wherein the child token bucket comprises a first category token bucket corresponding to the first category of file interactions and wherein accessing a portion of the hierarchical token bucket structure comprises:
based on determining that there are not enough tokens in the child token bucket available to service the token request, accessing the parent token bucket to determine whether there are enough tokens available to service the token request.