Prefetch bandwidth throttling by dynamically adjusting miss buffer prefetch-dropping thresholds
The disclosed embodiments relate to a method for controlling prefetching in a processor to prevent over-saturation of interfaces in the memory hierarchy of the processor. While the processor is executing, the method determines a bandwidth utilization of an interface from a cache in the processor to a lower level of the memory hierarchy. Next, the method selectively adjusts a prefetch-dropping high-water mark for occupancy of a miss buffer associated with the cache based on the determined bandwidth utilization, wherein the miss buffer stores entries for outstanding demand requests and prefetches that missed in the cache and are waiting for corresponding data to be returned from the lower level of the memory hierarchy, and wherein when the occupancy of the miss buffer exceeds the prefetch-dropping high-water mark, subsequent prefetches that cause a cache miss are dropped.
1. A method for controlling prefetching to prevent over-saturation of interfaces in a memory hierarchy of a processor, comprising:
while the processor is executing, determining a bandwidth utilization of an interface from a cache in the processor to a lower level of the memory hierarchy; and
selectively adjusting a prefetch-dropping high-water mark for occupancy of a miss buffer associated with the cache based on the determined bandwidth utilization by:
selecting a lower prefetch-dropping high-water mark when the determined bandwidth utilization indicates that the interface from the cache to the lower level of the memory hierarchy has become over-saturated; and
selecting a higher prefetch-dropping high-water mark when the determined bandwidth utilization indicates that the interface from the cache to the lower level of the memory hierarchy has become unsaturated;
wherein the prefetch-dropping high-water mark is selectively adjusted to provide different degrees of throttling prefetches generated by a prefetcher;
wherein the miss buffer stores entries for outstanding demand requests and prefetches that missed in the cache and are waiting for corresponding data to be returned from the lower level of the memory hierarchy, wherein the demand requests do not include prefetches; and
wherein when an occupancy of the miss buffer exceeds the prefetch-dropping high-water mark, subsequent prefetches, generated by the prefetcher, that cause a cache miss are dropped, while subsequent demand requests are stored in the miss buffer.
2. The method of claim 1 , wherein determining the bandwidth utilization of the interface comprises determining an average latency involved in processing requests that missed in the cache over a preceding time period.
3. The method of claim 1 , wherein determining the bandwidth utilization of the interface includes determining a number of requests that missed in the cache and were sent to the lower level of the memory hierarchy over a preceding time period.
4. The method of claim 1 , wherein determining the bandwidth utilization of the interface includes:
obtaining information from the lower level of the memory hierarchy about occupancy levels for input queues associated with the lower level of the memory hierarchy; and
using the obtained occupancy information for the input queues associated with the lower level of the memory hierarchy to deduce the bandwidth utilization for the interface between the cache and the lower level of the memory hierarchy.
5. The method of claim 1 , wherein the prefetch-dropping high-water mark is adjusted among more than two possible values.
6. The method of claim 1 , wherein each prefetch-dropping high-water mark is associated with a different type of prefetch; and wherein different types of prefetches include hardware prefetches and software prefetches.
7. The method of claim 1 , wherein the prefetch-dropping high-water mark is adjusted based on bandwidth utilizations for one or more interfaces among different levels of the memory hierarchy.
8. The method of claim 1 , wherein the lower level of the memory hierarchy comprises one of: a lower level cache; and a system memory.
9. The method of claim 1 , wherein the cache comprises one of: an L3 cache; an L2 cache; and an L1 cache.
10. A processor with a mechanism that controls prefetching to prevent over-saturation of interfaces in a memory hierarchy of the processor, comprising:
a processor core;
a single or multilevel cache that is part of the memory hierarchy;
bandwidth-determination logic that determines a bandwidth utilization of an interface from the cache to a lower level of the memory hierarchy; and
a miss buffer associated with the cache that stores entries for outstanding demand requests and prefetches that missed in the cache and are waiting for corresponding data to be returned from the lower level of the memory hierarchy, wherein the demand requests do not include prefetches;
wherein the prefetch-dropping high-water mark is selectively adjusted to provide different degrees of throttling prefetches generated by a prefetcher;
wherein the miss buffer selectively adjusts a prefetch-dropping high-water mark for occupancy of the miss buffer based on the determined bandwidth utilization by:
selecting a lower prefetch-dropping high-water mark when the determined bandwidth utilization indicates that the interface from the cache to the lower level of the memory hierarchy has become over-saturated; and
selecting a higher prefetch-dropping high-water mark when the determined bandwidth utilization indicates that the interface from the cache to the lower level of the memory hierarchy has become unsaturated; and
wherein when an occupancy of the miss buffer exceeds the prefetch-dropping high-water mark, subsequent prefetches, generated by the prefetcher, that cause a cache miss are dropped, while subsequent demand requests are stored in the miss buffer.
11. The processor of claim 10 , wherein while determining the bandwidth utilization of the interface, the bandwidth-determination logic determines an average latency involved in processing requests that missed in the cache over a preceding time period.
12. The processor of claim 10 , wherein while determining the bandwidth utilization of the interface, the bandwidth-determination logic determines a number of requests that missed in the cache and were sent to the lower level of the memory hierarchy over a preceding time period.
13. The processor of claim 10 , wherein while determining the bandwidth utilization of the interface, the bandwidth-determination logic:
obtains information from the lower level of the memory hierarchy about occupancy levels for input queues associated with the lower level of the memory hierarchy; and
uses the obtained occupancy information for the input queues associated with the lower level of the memory hierarchy to deduce the bandwidth utilization for the interface between the cache and the lower level of the memory hierarchy.
14. The processor of claim 10 , wherein the miss buffer adjusts the prefetch-dropping high-water mark among more than two possible values.
15. The processor of claim 10 , wherein each prefetch-dropping high-water mark is associated with a different type of prefetch; and wherein different types of prefetches include hardware prefetches and software prefetches.
16. The processor of claim 10 , wherein the miss buffer adjusts the prefetch-dropping high-water mark based on bandwidth utilizations for one or more interfaces among different levels of the memory hierarchy.
17. The processor of claim 10 , wherein the cache comprises one of: an L3 cache; an L2 cache; and an L1 cache.
18. A computer system, comprising:
a processor; and
a memory hierarchy, which includes a cache hierarchy and a memory;
wherein the processor comprises,
a processor core,
the cache hierarchy that includes one or more caches;
bandwidth-determination logic that determines a bandwidth utilization of an interface from the cache to a lower level of the memory hierarchy, and
a miss buffer associated with the cache that stores entries for outstanding demand requests and prefetches that missed in the cache and are waiting for corresponding data to be returned from the lower level of the memory hierarchy, wherein demand requests do not include prefetches,
wherein the prefetch-dropping high-water mark is selectively adjusted to provide different degrees of throttling prefetches generated by a prefetcher;
wherein the miss buffer selectively adjusts a prefetch-dropping high-water mark for occupancy of the miss buffer based on the determined bandwidth utilization by:
selecting a lower prefetch-dropping high-water mark when the determined bandwidth utilization indicates that the interface from the cache to the lower level of the memory hierarchy has become over-saturated; and
selecting a higher prefetch-dropping high-water mark when the determined bandwidth utilization indicates that the interface from the cache to the lower level of the memory hierarchy has become unsaturated, and
wherein when an occupancy of the miss buffer exceeds the prefetch-dropping high-water mark, subsequent prefetches, generated by the prefetcher, that cause a cache miss are dropped, while subsequent demand requests are stored in the miss buffer.