Method and apparatus for reduction of recurring cache misses
A control circuit is configured to receive a stream of addresses associated with cache lookup requests resulting from execution of a workload by a data processing system. The control circuit includes a cache miss monitor and a cache hit monitor both coupled to a controller. The cache miss monitor determines a number of reoccurring misses as a difference between the number of addresses in the stream of addresses that miss in the cache and the number of such addresses that are unique. The cache hit monitor includes a moment circuit configured to determine first and second frequency moments addresses in the stream of addresses that hit in the cache. The controller controls the data processing system based on the number of reoccurring misses and the first and second frequency moments. A bypass mechanism of the cache may be controlled, for example.
1 . A control circuit configured to receive a stream of addresses associated with cache lookup requests to a cache of a data processing system, the requests resulting from execution of a workload by the data processing system, and the control circuit comprising:
a cache miss monitor including:
a first counter configured to determine a number of addresses in the stream of addresses for which a corresponding lookup request resulted in a miss in the cache;
a cardinality circuit configured to determine a number of unique addresses in the stream of addresses for which a corresponding lookup request resulted in a miss in the cache; and
a circuit configured to determine a number of reoccurring misses as a difference between the number of addresses and the number of unique addresses;
a cache hit monitor including:
a moment circuit configured to determine a first frequency moment μ h and a second frequency moment M h (2) of addresses in the stream of addresses for which a corresponding lookup request resulted in a hit in the cache; and
a controller configured to control the data processing system based on the number of reoccurring misses, the first frequency moment μ h and the second frequency moment M h (2) .
2 . The control circuit of claim 1 , where the cardinality circuit includes a circuit configured to determine a statistical estimate of the number of unique addresses in a block of addresses of the stream of addresses and the moment circuit is configured to determine statistical estimates of the first and second frequency moments over blocks of addresses of the stream of addresses.
3 . The control circuit of claim 1 , where the cache hit monitor includes: an allocation circuit configured to allocate one or more addresses of the stream of addresses in a memory; a second counter configured to count a number of addresses in the stream of addresses for which the address matches an address allocated in the memory and a corresponding lookup request resulted in a hit in the cache; and a third counter configured to count a number of allocations made by the allocation circuit.
4 . The control circuit of claim 1 , further comprising an estimation circuit configured to determine a cache-thrashing indicator based on the number of reoccurring misses, the first frequency moment μ h and the second frequency moment M h (2) , where the controller is configured to control the data processing system based on the cache-thrashing indicator.
5 . The control circuit of claim 4 , where the cache-thrashing indicator is also based on a ratio WIT, where T is the number of addresses in the stream of addresses for which a corresponding lookup request resulted in a miss in the cache and W is the number of reoccurring misses in the address stream.
6 . The control circuit of claim 4 , further comprising a cache bypass circuit configured to:
randomly bypass the cache for a cache miss with a probability based on the cache-thrashing indicator;
bypass the cache for a cache miss for a data address based on the data address and the cache-thrashing indicator; or
bypass the cache for a cache miss for a data address based on a hash of the data address and the cache-thrashing indicator.
7 . The control circuit of claim 4 , further comprising:
a working set size estimator configured to monitor the address stream and generate a statistical estimate of a size of a working set of the cache,
where the controller is further configured to control a bypass mechanism of the cache to:
bypass allocations in the cache based on the number of reoccurring misses, the first frequency moment μ h and the second frequency moment M h (2) ; and
bypass the cache based on the statistical estimate of the size of the working set of the cache.
8 . The control circuit of claim 1 , where the cache hit monitor includes:
a memory configured to store a set of R frequency counts, where a frequency count c(i) at location i in the memory is associated with an address a(i) in a set of addresses selected from the stream of addresses;
a replacement circuit configured to:
replace an address in the set of addresses with a current address in the stream of addresses based on a designated probability, and
set frequency count c(i) to one when the address associated with frequency count c(i) addresses is replaced;
a memory update circuit configured to increment the frequency count c(i) stored in the memory when an associated addresses a(i) in the set of addresses occurs in the stream of cache lookup addresses and a corresponding cache lookup is a miss in the cache; and
an accumulator update circuit configured to update one or more accumulator values based on the frequency counts;
where the moment circuit is configured to determine the first and second frequency moments based on the accumulated values.
9 . The control circuit of claim 8 , where the accumulator update circuit is configured to:
when an address a(i) in the set of addresses occurs in the stream of cache lookup addresses and the corresponding cache lookup is a miss in the cache:
increment a first accumulator value F 1 , and
add 2×c(i)−1 to a second accumulator value F 2 , and
increment a sample accumulator value F 0 when an address in the set of addresses is replaced.
10 . The control circuit of claim 9 , where the moment circuit is configured to determine a second frequency moment M h (2) of the address stream as F 2 /F 0 .
11 . The control circuit of claim 8 , where the memory includes a content address memory (CAM) configured to store the set of addresses, the CAM determining an index i in response to an address a(i).
12 . The control circuit of claim 1 , where the cardinality circuit includes: a memory, configured to store maximum pattern lengths; match circuitry configured to generate a first pattern match length of a first bit vector, the first bit vector corresponding to an address of the stream of addresses or a hash of an address of the stream of addresses; memory update circuitry configured to: receive a first index; compare the first pattern match length to a maximum pattern length for the first index stored in the memory; and update the maximum pattern match length for the first index in the memory with the first pattern match length when the first pattern match length is greater than the maximum pattern length for the first index; sum update circuitry configured to update a first summation value based on the maximum pattern match length for the first index and the first pattern match length when the first pattern match length is greater than the maximum pattern length for the first index; and sequential logic circuitry configured to generate a first statistical cardinality count from a scaled reciprocal of the first summation, the first statistical cardinality count indicating number of unique addresses in the stream of addresses for which a corresponding lookup request resulted in a miss in the cache.
13 . The control circuit of claim 12 , further comprising hash circuitry configured to generate a hash value from an address of the stream of addresses, the hash value including the first index and the first bit vector.
14 . The control circuit of claim 1 , where the controller is configured to control a bypass mechanism of the cache, a partitioning of the cache, or an allocation of the workload.
15 . The control circuit of claim 1 , where the moment circuit of the cache hit monitor is configured to determine the first frequency moment μ h from the stream of addresses and to determine the second frequency moment M h (2) from the stream of addresses or from the first frequency moment and a variance parameter on.
16 . A computer-implemented method comprising, for a sequence of cache requests in a data processing system having a cache, the sequence of cache requests resulting from execution of a workload by the data processing system:
determining an average reoccurring miss frequency μ m of addresses in the sequence of cache requests based on a total number of addresses in the sequence and a number of unique addresses in the sequence;
determining a first frequency moment μ h and a second frequency moment M h (2) of sample addresses in the sequence of the cache requests that resulted in a hit in the cache; and
controlling the data processing system based on the average reoccurring miss frequency μ m , the first frequency moment μ h and the second frequency moment M h (2) .
17 . The computer-implemented method of claim 16 , where determining the average reoccurring miss frequency μ m includes:
estimating an average number of misses between new reoccurring misses; and
determining the average reoccurring miss frequency μ m as a function of the average number of misses between new reoccurring misses.
18 . The computer-implemented method of claim 16 , further comprising
increasing a cache-thrashing indicator based on the average reoccurring miss frequency μ m , and
decreasing the cache-thrashing indicator based on the first frequency moment μ h and a second frequency moment M h (2) ;
where controlling the data processing system includes controlling at least one of:
a bypass mechanism of the cache,
a partitioning of the cache, and
allocation of the workload,
based on the cache-thrashing indicator.
19 . The computer-implemented method of claim 18 , where decreasing the cache-thrashing indicator based on the first frequency moment μ h and the second frequency moment M h (2) includes determining a probability that a reoccurring miss can be allocated in the cache in place of a low activity or dead cache block.
20 . A non-transitory computer-readable medium storing computer-readable code for fabrication of a control circuit comprising:
a cache miss monitor including:
a first counter configured to determine a number of addresses in a stream of addresses associated with cache lookup requests to a cache for which a corresponding lookup request resulted in a miss in the cache, the lookup requests resulting from execution of a workload in a data processing system;
a cardinality circuit configured to determine a number of unique addresses in the stream of addresses for which a corresponding lookup request resulted in a miss in the cache; and
a circuit configured to determine a number of reoccurring misses as a difference between the number of addresses and the number of unique addresses;
a cache hit monitor including:
a moment circuit configured to determine a first frequency moment μ h and a second frequency moment M h (2) of addresses in the stream of addresses for which a corresponding lookup request resulted in a hit in the cache; and
an estimation circuit configured to determine a cache-thrashing indicator based on the number of reoccurring misses, the first frequency moment μ h and the second frequency moment M h (2) .