Systems and methods for adaptive hybrid hardware pre-fetch
An apparatus includes a processor core and a memory hierarchy. The memory hierarchy includes main memory and one or more caches between the main memory and the processor core. A plurality of hardware pre-fetchers are coupled to the memory hierarchy and a pre-fetch control circuit is coupled to the plurality of hardware pre-fetchers. The pre-fetch control circuit is configured to compare changes in one or more cache performance metrics over two or more sampling intervals and control operation of the plurality of hardware pre-fetchers in response to a change in one or more performance metrics between at least a first sampling interval and a second sampling interval.
1. A method comprising:
operating a plurality of hardware pre-fetchers to pre-fetch data in a memory hierarchy that includes a main memory and two or more caches between the main memory and a processor core, each hardware pre-fetcher coupled to each of the two or more caches to apply a different pre-fetching scheme to the two or more caches;
arbitrating requests from each of the plurality of hardware pre-fetchers;
buffering pre-fetch requests from each of the plurality of hardware pre-fetchers in a buffer;
calculating changes in one or more performance metrics over two or more sampling intervals; and
controlling the plurality of hardware pre-fetchers in response to a change in the one or more performance metrics between at least a first sampling interval and a second sampling interval.
2. The method of claim 1 wherein the change in the one or more performance metrics includes a change in pre-fetch accuracy between at least the first sampling interval and the second sampling interval.
3. The method of claim 1 , wherein controlling the plurality of hardware pre-fetchers in response to a change in the one or more performance metrics includes reducing aggressiveness of a hardware pre-fetcher in response to a reduction in pre-fetch accuracy from the first sampling interval to the second sampling interval.
4. The method of claim 3 , further comprising controlling the plurality of hardware pre-fetchers in response to a detected level of system utilization including reducing aggressiveness of the hardware pre-fetcher in response to high system utilization.
5. The method of claim 3 , wherein reducing aggressiveness of the hardware pre-fetcher includes reducing at least one of degree or distance used by the hardware pre-fetcher.
6. The method of claim 1 further comprising:
sending a first control signal to a first hardware pre-fetcher of the plurality of hardware pre-fetchers;
sending a second control signal to a second hardware pre-fetcher of the plurality of hardware pre-fetchers; and
sending a third control signal to a third hardware pre-fetcher of the plurality of hardware pre-fetchers.
7. The method of claim 6 wherein the first hardware pre-fetcher is a Multi-Offset Pre-fetcher (MOP), the second hardware pre-fetcher is a Spatial Memory Streaming (SMS) pre-fetcher, and the third hardware pre-fetcher is an Enhanced Next-line Pre-fetcher (ENP).
8. The method of claim 1 further comprising:
counting evictions from a cache of the memory hierarchy, the first and second sampling intervals defined by a predetermined number of evictions from the cache.
9. The method of claim 1 wherein the one or more caches between the main memory and the processor core include a Level 1 (L1) cache and a Level 2 (L2) cache, and calculating changes in one or more performance metrics includes calculating a first change in a performance metric for the L1 cache and calculating a second change in the performance metric for the L2 cache.
10. An apparatus comprising:
a processor core;
a memory hierarchy comprising main memory and two or more caches between the main memory and the processor core;
a plurality of hardware pre-fetchers, each hardware pre-fetcher coupled to each of the two or more caches of the memory hierarchy, each hardware pre-fetcher configured to apply a different pre-fetching scheme to the two or more caches of the memory hierarchy;
an arbiter coupled to the plurality of hardware pre-fetchers to arbitrate requests from each of the plurality of hardware pre-fetchers;
a buffer coupled to the plurality of hardware pre-fetchers through the arbiter to enqueue pre-fetch requests from each of the plurality of hardware pre-fetchers; and
a pre-fetch control circuit coupled to the plurality of hardware pre-fetchers, the pre-fetch control circuit configured to measure changes in one or more cache performance metrics over two or more sampling intervals and to control operation of the plurality of hardware pre-fetchers in response to a change in one or more performance metrics between at least a first sampling interval and a second sampling interval.
11. The apparatus of claim 10 , wherein the change in the one or more performance metrics includes detecting a change in pre-fetch accuracy between at least the first sampling interval and the second sampling interval.
12. The apparatus of claim 10 , wherein the pre-fetch control circuit is configured to reduce aggressiveness of a hardware pre-fetcher in response to a reduction in pre-fetch accuracy from the first sampling interval to the second sampling interval.
13. The apparatus of claim 12 wherein the pre-fetch control circuit is further configured to control operation of the plurality of hardware pre-fetchers in response to a detected level of system utilization, the pre-fetch control circuit configured to reduce aggressiveness of the hardware pre-fetcher in response to high system utilization.
14. The apparatus of claim 12 , wherein the pre-fetch control circuit is configured to reduce aggressiveness of the hardware pre-fetcher by reducing at least one of degree or distance used by the hardware pre-fetcher.
15. The apparatus of claim 10 , wherein the plurality of hardware pre-fetchers includes a first hardware pre-fetcher, a second hardware pre-fetcher and a third hardware pre-fetcher, each connected to receive a separate control signal from the pre-fetch control circuit.
16. The apparatus of claim 15 , wherein the first hardware pre-fetcher is a Multi-Offset Pre-fetcher (MOP), the second hardware pre-fetcher is a Spatial Memory Streaming (SMS) pre-fetcher, and the third hardware pre-fetcher is an Enhanced Next-line Pre-fetcher (ENP).
17. The apparatus of claim 10 , wherein the first and second sampling intervals are each defined by a predetermined number of evictions in a corresponding cache.
18. The apparatus of claim 10 , wherein the one or more caches between the main memory and the processor core include a Level 1 (L1) cache and a Level 2 (L2) cache, each of the plurality of hardware pre-fetchers operating separately on L1 cache and L2 cache according to separate performance metrics and separate sampling intervals.
19. The apparatus of claim 10 , wherein the plurality of hardware pre-fetchers includes a Multi-Offset Prefetcher (MOP).
20. The apparatus of claim 10 , wherein the plurality of hardware pre-fetchers includes a Spatial Memory Streaming (SMS) pre-fetcher.
21. The apparatus of claim 10 , wherein the plurality of hardware pre-fetchers includes an Enhanced Next-line Pre-fetcher (ENP).
22. A computer system comprising:
a processor core;
a memory hierarchy comprising main memory and at least a first cache and a second cache between the main memory and the processor core;
a plurality of hardware pre-fetchers, each hardware pre-fetcher coupled to the first cache and the second cache of the memory hierarchy, each hardware pre-fetcher configured to apply a different pre-fetching scheme to the two or more caches of the memory hierarchy;
an arbiter coupled to the plurality of hardware pre-fetchers to arbitrate requests from each of the plurality of hardware pre-fetchers;
a buffer coupled to the plurality of hardware pre-fetchers through the arbiter to enqueue pre-fetch requests from each of the plurality of hardware pre-fetchers; and
a pre-fetch control circuit coupled to the plurality of hardware pre-fetchers, the pre-fetch control circuit configured to calculate changes in pre-fetch accuracy of the first cache and the second cache over two or more sampling intervals and control aggressiveness of the plurality of hardware pre-fetchers in response to changes in pre-fetch accuracy of a corresponding cache between at least a first sampling interval and a second sampling interval of the corresponding cache.
23. The computer system of claim 22 , wherein the pre-fetch control circuit is configured to reduce aggressiveness of a hardware pre-fetcher in response to a reduction in pre-fetch accuracy from the first sampling interval to the second sampling interval and to increase aggressiveness of the hardware pre-fetcher in response to an increase in pre-fetch accuracy of the hardware pre-fetcher from the first sampling interval to the second sampling interval.
24. The computer system of claim 22 , wherein the plurality of hardware pre-fetchers includes a Multi-Offset Prefetcher (MOP).
25. The computer system of claim 22 , wherein the plurality of hardware pre-fetchers includes a Spatial Memory Streaming (SMS) pre-fetcher.
26. The computer system of claim 22 , wherein the plurality of hardware pre-fetchers includes an Enhanced Next-line Pre-fetcher (ENP).
27. The computer system of claim 22 , wherein the plurality of hardware pre-fetchers includes a Multi-Offset Prefetcher (MOP), a Spatial Memory Streaming (SMS) pre-fetcher and an Enhanced Next-line Pre-fetcher (ENP).