Fine-grained multi-tenant cache management
The embodiments herein describe a multi-tenant cache that implements fine-grained allocation of the entries within the cache. Each entry in the cache can be allocated to a particular tenant—i.e., fine-grained allocation—rather than having to assign all the entries in a way to a particular tenant. If the tenant does not currently need those entries (which can be tracked using counters), the entries can be invalidated (i.e., deallocated) and assigned to another tenant. Thus, fine-grained allocation provides a flexible allocation of entries in a hardware cache that permits an administrator to reserve any number of entries for a particular tenant, but also permit other tenants to use this bandwidth when the reserved entries are not currently needed by the tenant.
1. A computing system comprising:
a cache comprising a plurality of entries, wherein a first number of the plurality of entries is reserved for a first tenant and a second number of the plurality of entries is reserved for a second tenant, wherein the first number and the second number are a guaranteed minimum number of entries in the cache that must be allocated to the first and second tenant, respectively, when requested by those tenants, wherein the cache is configured to:
receive a request to allocate an entry of the cache, wherein the request corresponds to the first tenant,
upon determining a counter corresponding to the first tenant has a value less than the first number, allocating the entry of the cache, wherein the allocated entry in the cache stores a tenant ID corresponding to the first tenant and the first number is a maximum value of the counter, and
incrementing the counter.
2. The computing system of claim 1 , wherein plurality of entries are arranged in a plurality of ways, wherein each allocated entry in the plurality of entries stores a respective tenant ID.
3. The computing system of claim 1 , wherein allocating the entry comprises:
selecting a previously allocated entry that stores a tenant ID corresponding to a different tenant than the first tenant, and
invaliding the previously allocated entry before allocating the entry.
4. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate an entry of the cache for the first tenant; and
upon determining the counter has a value equal to or greater than the first number, determine whether there is an unallocated entry in the cache.
5. The computing system of claim 4 , wherein the cache is configured to:
upon determining there is an unallocated entry:
allocate the unallocated entry to the first tenant; and
increment the counter corresponding to the first tenant.
6. The computing system of claim 4 , wherein the cache is configured to:
upon determining there are no unallocated entries remaining in the cache:
invalidate an old entry previously allocated to the first tenant in the cache; and
allocate the old entry to the first tenant.
7. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate a second entry of the cache for the first tenant;
determine that the counter corresponding to the first tenant is equal to or greater than its maximum value;
determine that an overflow counter corresponding to an overflow portion of the cache is equal to or greater than its maximum value;
invalidate an old entry in the cache previously allocated to the first tenant; and
allocate the old entry to the first tenant.
8. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate a second entry of the cache for the first tenant;
determine that the second entry corresponds to a full index way in the cache;
determine that none of the entries in the full index way are allocated to the first tenant;
determine that one entry in the entries in the full index way corresponds to a second tenant that has a second counter that is less than its maximum value;
invalidate the one entry;
allocate the one entry to the first tenant;
increment the counter corresponding to the first tenant; and
decrement the second counter corresponding to the second tenant.
9. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate a second entry of the cache for the first tenant;
determine that the second entry corresponds to a full index way in the cache;
determine that none of the entries in the full index way are allocated to the first tenant;
determine that one entry in the entries in the full index way corresponds to a second tenant that has a second counter that is at least equal to its maximum value;
determine that an overflow counter corresponding to an overflow portion of the cache is non-zero;
invalidate the one entry;
allocate the one entry to the first tenant;
increment the counter corresponding to the first tenant; and
decrement the overflow counter corresponding to the overflow portion.
10. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate a second entry of the cache for the first tenant;
determine that the counter corresponding to the first tenant is equal to or greater than its maximum value;
determine that an overflow counter corresponding to an overflow portion of the cache is less than its maximum value;
determine that the second entry corresponds to a full index way in the cache;
identify at least one entry in the full index way previously allocated to the first tenant;
invalidate the at least one entry; and
allocate the at least one entry to the first tenant.
11. A method comprising:
receiving a request to allocate an entry of a multi-tenant cache, wherein a first number of the entries in the multi-tenant cache is reserved for a first tenant and a second number of the entries in the multi-tenant cache is reserved for a second tenant, wherein the first number and the second number are a guaranteed minimum number of entries in the multi-tenant cache that must be allocated to the first and second tenant, respectively, when requested by those tenants, and wherein the request corresponds to the first tenant;
upon determining a counter corresponding to the first tenant has a value less than the first number, allocating the entry of the multi-tenant cache, wherein the allocated entry stores a tenant ID corresponding to the first tenant and the first number is a maximum value of the counter; and
incrementing the counter.
12. The method of claim 11 , wherein the entries are arranged in a plurality of ways, wherein each allocated entry in the entries stores a respective tenant ID.
13. The method of claim 12 , wherein allocating the entry comprises:
selecting a previously allocated entry that stores a tenant ID corresponding to a different tenant than the first tenant, and
invaliding the previously allocated entry before allocating the entry.
14. The method of claim 11 , further comprising:
receiving a second request to allocate a second entry of the multi-tenant cache for the first tenant; and
upon determining the counter has a value equal to or greater than the first number, determining whether there is an unallocated entry in the cache.
15. The method of claim 14 , further comprising:
upon determining there is an unallocated entry:
allocating the unallocated entry to the first tenant; and
incrementing the counter corresponding to the first tenant.
16. The method of claim 14 , further comprising:
upon determining there are no unallocated entries remaining in the cache:
invalidating an old entry previously allocated to the first tenant in the cache; and
allocating the old entry to the first tenant.
17. A computing system comprising:
a cache comprising a plurality of entries, wherein a first number of the plurality of entries is reserved for a first tenant and a second number of the plurality of entries is reserved for a second tenant, wherein the first number and the second number are a guaranteed minimum number of entries in the cache that must be allocated to the first and second tenant, respectively, when requested by those tenants, wherein the cache is configured to:
receive a request to invalidate an entry of the cache, wherein the request corresponds to the first tenant,
invalidate a first entry of the plurality of entries in the cache that was previously allocated to the first tenant, wherein the first entry in the cache stores a tenant ID corresponding to the first tenant, and
decrementing a counter corresponding to the first tenant, wherein the first number is a maximum value of the counter.
18. The computing system of claim 17 , wherein plurality of entries are arranged in a plurality of ways, wherein each allocated entry in the plurality of entries stores a respective tenant ID.
19. The computing system of claim 17 , wherein the cache is further configured to:
receive a second request to invalidate a second entry of the cache, wherein the request corresponds to the first tenant;
determine that the counter corresponding to the first tenant is less than the first number;
determine that an overflow counter corresponding to an overflow portion of the cache is non-zero;
invalidate, using at least the tenant ID, a first entry in the cache previously allocated to the first tenant; and
decrement the overflow counter.
20. The computing system of claim 17 , wherein the cache is further configured to:
receive a second request to invalidate a second entry of the cache, wherein the request corresponds to the first tenant;
determine that the counter corresponding to the first tenant is less than the first number;
determine that an overflow counter corresponding to an overflow portion of the cache is zero;
invalidate, using at least the tenant ID, a first entry in the cache previously allocated to the first tenant; and
decrement the counter corresponding to the first tenant.