Fine-grained multi-tenant cache management
The embodiments herein describe a multi-tenant cache that implements fine-grained allocation of the entries within the cache. Each entry in the cache can be allocated to a particular tenant—i.e., fine-grained allocation—rather than having to assign all the entries in a way to a particular tenant. If the tenant does not currently need those entries (which can be tracked using counters), the entries can be invalidated (i.e., deallocated) and assigned to another tenant. Thus, fine-grained allocation provides a flexible allocation of entries in a hardware cache that permits an administrator to reserve any number of entries for a particular tenant, but also permit other tenants to use this bandwidth when the reserved entries are not currently needed by the tenant.
1. A computing system comprising:
a cache comprising a plurality of entries, wherein a first number of the plurality of entries is reserved for a first tenant and a second number of the plurality of entries is reserved for a second tenant, wherein the first number and the second number are a guaranteed minimum number of entries in the cache that must be allocated to the first and second tenant, respectively, when requested by those tenants, wherein the cache is configured to:
receive a request to allocate an entry of the cache, wherein the request comprises a first tenant ID corresponding to the first tenant,
determine that a counter corresponding to the first tenant has a value equal to or greater than the first number and the request comprises the first tenant ID, determine that an overflow counter corresponding to an overflow portion of the cache is equal to or greater than its maximum value,
invalidate an old entry in the cache previously allocated,
allocate the old entry to the first tenant, and increment the counter.
2. The computing system of claim 1 , wherein the plurality of entries are arranged in a plurality of ways, wherein each allocated entry in the plurality of entries stores a respective tenant ID.
3. The computing system of claim 1 , wherein allocating the old entry comprises:
selecting a previously allocated entry that stores a second tenant ID corresponding to a different tenant than the first tenant, and
invaliding the previously allocated entry before allocating the entry.
4. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate an entry of the cache for the first tenant; and
upon determining the counter has a value less than the first number:
allocate the entry of the cache to the first tenant; and
increment the counter.
5. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate an entry of the cache for the first tenant;
upon determining the counter has a value equal to or greater than the first number, determine whether there is an unallocated entry in the cache; and
upon determining there is an unallocated entry:
allocate the unallocated entry to the first tenant; and
increment the counter corresponding to the first tenant.
6. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate an entry of the cache for the first tenant;
upon determining the counter has a value equal to or greater than the first number, determine whether there is an unallocated entry in the cache; and
upon determining there are no unallocated entries remaining in the cache:
invalidate a previously allocated entry in the cache; and
allocate the previously allocated entry to the first tenant.
7. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate a second entry of the cache for the first tenant;
determine that the counter corresponding to the first tenant is equal to or greater than its maximum value;
determine that the overflow counter corresponding to an overflow portion of the cache is less than its maximum value;
determine that the entry corresponds to a full index way in the cache;
identify that at least one entry in the full index way has not been allocated; and
allocate the at least one entry to the first tenant, and
increment the overflow counter.
8. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate a second entry of the cache for the first tenant;
determine that the second entry corresponds to a full index way in the cache;
determine that none of the entries in the full index way are allocated to the first tenant;
determine that one entry in the entries in the full index way corresponds to a second tenant that has a second counter that is less than its maximum value;
invalidate the one entry;
allocate the one entry to the first tenant;
increment the counter corresponding to the first tenant; and
decrement the second counter corresponding to the second tenant.
9. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate a second entry of the cache for the first tenant;
determine that the second entry corresponds to a full index way in the cache;
determine that none of the entries in the full index way are allocated to the first tenant;
determine that one entry in the entries in the full index way corresponds to a second tenant that has a second counter that is at least equal to its maximum value;
determine that the overflow counter corresponding to the overflow portion of the cache is non-zero;
invalidate the one entry;
allocate the one entry to the first tenant;
increment the counter corresponding to the first tenant; and
decrement the overflow counter corresponding to the overflow portion.
10. The computing system of claim 1 , wherein the cache is configured to:
receive a second request to allocate a second entry of the cache for the first tenant;
determine that the counter corresponding to the first tenant is equal to or greater than its maximum value;
determine that the overflow counter corresponding to the overflow portion of the cache is less than its maximum value;
determine that the second entry corresponds to a full index way in the cache;
identify at least one entry in the full index way previously allocated to the first tenant;
invalidate the at least one entry; and
allocate the at least one entry to the first tenant.
11. A method comprising:
receiving a request to allocate an entry of a multi-tenant cache comprising a plurality of entries, wherein a first number of the plurality of entries is reserved for a first tenant and a second number of the plurality of entries is reserved for a second tenant, wherein the first number and the second number are a guaranteed minimum number of entries in the multi-tenant cache that must be allocated to the first and second tenant, respectively, when requested by those tenants, and wherein the request comprises a first tenant ID that corresponds to the first tenant;
determining a counter corresponding to the first tenant has a value equal to or greater than the first number and the request comprises the first tenant ID;
determining that an overflow counter corresponding to an overflow portion of the multi-tenant cache is equal to or greater than its maximum value;
invalidating an old entry in the multi-tenant cache previously allocated;
allocating the old entry to the first tenant; and
incrementing the counter.
12. The method of claim 11 , wherein the plurality of entries are arranged in a plurality of ways, wherein each allocated entry in the plurality of entries stores a respective tenant ID.
13. The method of claim 11 , wherein allocating the old entry comprises:
selecting a previously allocated entry that stores a second tenant ID corresponding to a different tenant than the first tenant, and
invaliding the previously allocated entry before allocating the entry.
14. The method of claim 11 , further comprising:
receiving a second request to allocate a second entry of the multi-tenant cache for the first tenant; and
upon determining the counter has a value less than the first number:
allocating the entry of the multi-tenant cache to the first tenant; and
incrementing the counter.
15. The method of claim 11 , further comprising:
receiving a second request to allocate an entry of the multi-tenant cache for the first tenant;
upon determining the counter has a value equal to or greater than the first number, determining whether there is an unallocated entry in the cache; and
upon determining there is an unallocated entry:
allocating the unallocated entry to the first tenant; and
incrementing the counter corresponding to the first tenant.
16. The method of claim 11 , further comprising:
receiving a second request to allocate an entry of the multi-tenant cache for the first tenant;
upon determining the counter has a value equal to or greater than the first number, determining whether there is an unallocated entry in the cache; and
upon determining there are no unallocated entries remaining in the multi-tenant cache:
invalidating a previously allocated entry in the multi-tenant cache; and
allocating the previously allocated entry to the first tenant.
17. A computing system comprising:
a cache comprising a plurality of entries, wherein a first number of the plurality of entries is reserved for a first tenant and a second number of the plurality of entries is reserved for a second tenant, wherein the first number and the second number are a guaranteed minimum number of entries in the cache that must be allocated to the first and second tenant, respectively, when requested by those tenants, wherein the cache is configured to:
receive a request to invalidate an entry of the cache, wherein the request corresponds to the first tenant,
determine that a counter corresponding to the first tenant is at the first number,
determine that an overflow counter corresponding to an overflow portion of the cache is non-zero,
invalidate a first entry of the plurality of entries in the cache that was previously allocated to the first tenant, and
decrement the overflow counter.
18. The computing system of claim 17 , wherein the plurality of entries are arranged in a plurality of ways, wherein each allocated entry in the plurality of entries stores a respective tenant ID.
19. The computing system of claim 17 , wherein the cache is further configured to:
receive a second request to invalidate a second entry of the cache, wherein the request corresponds to the first tenant;
determine that the counter corresponding to the first tenant is at the first number;
determine that an overflow counter corresponding to an overflow portion of the cache is zero;
invalidate a first entry in the cache previously allocated to the first tenant; and
decrement the counter corresponding to the first tenant.