Multi-layer multi-hit caching for long tail content
Some embodiments provide an optimized multi-hit caching technique that minimizes the performance impact associated with caching of long-tail content while retaining much of the efficiency and minimal overhead associated with first hit caching in determining when to cache content. The optimized multi-hit caching utilizes a modified bloom filter implementation that performs flushing and state rolling to delete indices representing stale content from a bit array used to track hit counts without affecting identification of other content that may be represented with indices overlapping with those representing the stale content. Specifically, a copy of the bit array is stored prior to flushing the bit array so as to avoid losing track of previously requested and cached content when flushing the bit array and the flushing is performed to remove the bit indices representing stale content from the bit array and to minimize the possibility of a false positive.
1. A method comprising:
tracking request counts for different content in (i) a first tier cache using different subsets of a plurality of indices in a first bloom filter and (ii) a second tier cache using different subsets of a plurality of indices in a second bloom filter, with each of said subset of indices in the first bloom filter tracking a request count for different content of a first plurality of content cached in the first tier cache and with each of said subset of indices in the second bloom filter tracking a request count for different content of a different second plurality of content cached in the second tier cache, and with at least one subset of indices tracking a request count greater than one;
receiving a request for particular content in the first tier cache;
identifying a particular subset of the plurality of indices in the first bloom filter with a minimum value of the particular subset of indices equaling M, wherein M is a value greater than one, and wherein M is the request count for the particular content, wherein said identifying comprises selecting the particular subset of indices from the plurality of indices based on a result of hashing the particular request;
moving said content from the first tier cache to the second tier cache in response to the minimum value of M from the particular subset of indices in the first bloom filter being greater than a value N setting a count threshold for entry into the second tier cache; and
transitioning, in response to said moving, a current request count for the particular content from the first bloom filter to the second bloom filter by incrementing each index of the particular subset of indices in the second bloom filter to at least M.
2. The method of claim 1 , wherein said transitioning comprises decrementing each index of the particular subset of indices in the first bloom filter by M.
3. The method of claim 1 further comprising purging said content from the first tier cache in response to said moving.
4. The method of claim 1 , wherein said tracking comprises hashing at least one identifier associated with the different content and incrementing different subsets of indices resulting from said hashing in the first bloom filter for content cached in the first tier cache and in the second bloom filter for content cached in the second tier cache.
5. The method of claim 1 further comprising discovering one of a plurality of cache tiers in which the particular content is located based on a minimum value of the particular subset of indices in the second bloom filter in place of accessing the first tier cache or the second tier cache.
6. The method of claim 5 , wherein said discovering is further based on the M value from the particular subset of indices in the first bloom filter and the minimum value of the particular subset of indices in the second bloom filter equaling a value of zero.
7. The method of claim 6 , wherein said discovering further comprises locating the particular content in the second tier cache in response to the minimum value of the particular subset of indices in the second bloom filter equaling a value of one or greater.
8. The method of claim 7 , wherein said discovering further comprises locating the particular content in the first tier cache in response to the minimum value of the particular subset of indices in the first bloom filter equaling a value of one or greater.
9. The method of claim 1 , wherein the particular subset of indices comprises two or more non-consecutive indices of the first and second bloom filters.
10. A method comprising:
tracking request counts for different content in (i) a first tier cache using different subsets of a plurality of indices in a first array and (ii) a second tier cache using different subsets of a plurality of indices in a second array;
receiving a request for particular content;
hashing an identifier associated with the particular content, wherein said hashing identifies a particular subset of the plurality of indices from the first and second arrays;
identifying a minimum value of M from the particular subset of indices from the first array and a minimum value of N from the particular subset of indices from the second array;
detecting a cached copy of the particular content to be in one of the first tier cache or the second tier cache based on said M and N values and without querying or accessing the first tier cache or the second tier cache, wherein said detecting places the cached copy in the second tier cache in response to N equaling a value greater than one and M, and wherein said detecting further places the cached copy in the first tier cache in response to N equaling a value of zero and M equaling a value of one or greater;
serving the cached copy of the particular content from the second tier cache in response to N equaling a value of two or greater and a value greater than M; and
serving the cached copy of the particular content from the first tier cache in response to M equaling a value of one or greater and N equaling zero.
11. The method of claim 10 further comprising retrieving a copy of the particular content from an external source in response to N and M equaling zero.
12. The method of claim 10 further comprising transitioning the cached copy of the particular content from the first tier cache to the second tier cache in response to M equaling a threshold value greater than one.
13. The method of claim 12 , wherein said transitioning comprises writing the cached copy of the particular content to the second tier cache and removing the cached copy of the particular content from the first tier cache.
14. The method of claim 12 , wherein said transitioning comprises incrementing each index of the particular subset of indices in the second array to at least M.
15. The method of claim 14 , wherein said transitioning further comprises decrementing each index of the particular subset of indices in the first array by M.
16. The method of claim 10 , wherein said locating comprises simultaneously querying the particular subset of indices from the first array and the second array.
17. The method of claim 10 further comprising decrementing the particular subset of indices from at least one of the first array or the second array in response to not receiving said request after a defined timer interval.
18. The method of claim 10 further comprising caching a first set of content requested less than N times in the first tier cache and caching a different second set of content requested more than N times in the second tier cache.
19. A caching server comprising:
a first storage cache caching a first set of content requested at least N times;
a second storage cache caching a different second set of content requested at least M times, wherein N is an integer value that is greater than M and greater than one, and wherein the second storage cache is slower than the first storage cache;
a memory configured with (i) a first array tracking request counts for the first set of content via different subsets of a plurality of indices in the first array and (ii) a second array tracking request counts for the second set of content via different subsets of a plurality of indices in the second array; and
a processor computing a position for where a cached copy of particular content is located, the position being in one of the first storage cache and the second storage cache, from (i) a first minimum value of a particular subset of indices in the first array of said memory and (ii) a second minimum value of the particular subset of indices in the second array of said memory, wherein the processor computes said position in the first storage cache without accessing the first storage cache in response to the first minimum value equaling at least N, wherein the processor computes said position in the second storage cache without accessing the second storage cache in response to the first minimum value being less than N and the second minimum value equaling at least M, wherein the particular subset of indices uniquely identifies the particular content from other content in the first set of content and the second set of content.
20. The caching server of claim 19 further comprising a network interface receiving a request comprising an identifier identifying the particular content, and wherein the processor, in response to said request, produces the particular subset of indices from hashing the identifier of the request.