Hardware guided software prefetch insertion
Generally disclosed herein is an approach for identifying target code locations for software prefetching instructions based on hardware prefetching information. The target code locations can be identified based on the locations where the hardware prefetchers were designed to prefetch loads. Hardware cache events related to processor events for different caches can be monitored for cache misses in a system running when hardware prefetchers are active and inactive. Code locations that may benefit from the use of the hardware prefetchers may be determined based on an ablation study. The identified code locations may be treated as recognized memory access patterns. Software prefetchers can be configured to use these memory access patterns for prefetching data while avoiding an extended time required to identify memory access patterns (i.e. a warm-up period) and fetching in locations where hardware prefetching is beneficial.
1. A system for hardware-guided software prefetch insertions, the system comprising:
one or more memories;
one or more processors in communication with the one or more memories, the one or more processors configured to:
receive first information related to a first hardware event while a hardware prefetcher is active;
disable the hardware prefetcher;
receive second information related to a second hardware event while the hardware prefetcher is disabled;
compare the first information with the second information;
determine one or more target code locations based on the comparison; and
generate software prefetching instructions for the target code locations.
2. The system of claim 1 , wherein the first hardware event and the second hardware event are monitored using telemetries.
3. The system of claim 1 , wherein the first hardware event and the second hardware event comprise memory bandwidth saturation, cache misses, and memory latency.
4. The system of claim 3 , wherein the first information and the second information are compared using metrics computed based on measurements of the memory bandwidth saturation, the cache misses, and the memory latency.
5. The system of claim 1 , wherein the one or more target code locations are identified based on target functions.
6. The system of claim 5 , wherein the target functions include prefetch-friendly functions and prefetch-unfriendly functions.
7. The system of claim 6 , wherein the prefetch-friendly functions include one or more functions that regress in performance due to an increase in last-level cache (LLC) misses in response to the hardware prefetcher being disabled.
8. The system of claim 6 , wherein the prefetch-unfriendly functions include one or more functions that gain performance and decrease CPU cycles due to a lower memory latency in response to the hardware prefetcher being disabled.
9. The system of claim 1 , wherein the software prefetch insertion is generated based on one or more parameters selected from a group comprising a prefetch address, a prefetch distance, and a prefetch degree.
10. The system of claim 1 , wherein the one or more target code locations are sent to a user for a manual modification of one or more codes associated with the one or more target code locations.
11. The system of claim 1 , wherein the one or more processors are configured to automatically modify one or more codes associated with the one or more target code locations.
12. A method for hardware-guided software prefetch insertions, the method comprising:
receiving, by one or more processors, first information related to a first hardware event while a hardware prefetcher is active;
disabling, by the one or more processors, the hardware prefetcher;
receiving, by the one or more processors, second information related to a second hardware event while the hardware prefetcher is disabled;
comparing, by the one or more processors, the first information with the second information;
determining, by the one or more processors, one or more target code locations based on the comparison; and
generating, by the one or more processors, software prefetch instructions for the target code locations.
13. The method of claim 12 , wherein the first hardware event and the second hardware event are monitored using telemetries.
14. The method of claim 12 , wherein the first hardware event and the second hardware event comprise memory bandwidth saturation, cache misses, and memory latency.
15. The method of claim 14 , wherein the first information and the second information are compared using metrics computed based on measurements of the memory bandwidth saturation, the cache misses, and the memory latency.
16. The method of claim 12 , wherein the one or more target code locations are identified based on target functions.
17. The method of claim 16 , wherein the target functions include prefetch-friendly functions and prefetch-unfriendly functions.
18. The method of claim 17 , wherein the prefetch-friendly functions include one or more functions that regress in performance due to an increase in last-level cache (LLC) misses in response to the hardware prefetcher being disabled.
19. The method of claim 17 , wherein the prefetch-unfriendly functions include one or more functions that gain performance and decrease CPU cycles due to a lower memory latency in response to the hardware prefetcher being disabled.
20. The method of claim 12 , wherein the software prefetch insertion is generated based on one or more parameters selected from a group comprising a prefetch address, a prefetch distance, and a prefetch degree.