Application and processor guided memory prefetching
Devices and systems having memory-side adaptive prefetch decision-making, including associated methods, are disclosed and described. Adaptive information can be provided to memory-side controller and prefetch components that allow such memory-side components to prefetch data in a manner that is adaptive with respect to a particular read memory request or to a thread performing read memory requests.
1. A device, comprising:
a memory-side (MS) device, further comprising;
a nonvolatile memory (NVM) configured as system memory;
a MS cache communicatively coupled to the NVM and operable to store a cached subset of the NVM, the MS cache including volatile memory; and
a MS controller communicatively coupled to the NVM, to the MS cache, and configured to couple to a memory controller of a processor-side device, the MS controller including MS controller circuitry configured to;
retrieve read data from a memory address in the NVM to fill a read request from the memory controller;
check for a correlation between the read request and other data in the NVM;
retrieve prefetch data having the correlation with the read request; and
store the prefetch data in the MS cache.
2. The device of claim 1 , wherein the MS controller circuitry is further configured to discard any prefetch data retrieved with the read data if a correlation is not found.
3. The device of claim 1 , wherein the MS controller circuitry further comprises a prefetch engine that, to retrieve the prefetch data having the correlation with the read request, is further configured to:
determine a prefetch pattern from received adaptive information;
identify the prefetch data from the prefetch pattern; and
retrieve the prefetch data from the NVM according to the prefetch pattern.
4. The device of claim 3 , wherein the adaptive information comprises a thread identification (TID) of a hardware thread sending the read request.
5. The device of claim 4 , wherein the adaptive information further comprises a prefetch hint.
6. The device of claim 5 , wherein the prefetch hint includes a function that correlates the prefetch data with the read data, the memory address, or both.
7. The device of claim 5 , wherein the prefetch hint includes a priority indicator.
8. The device of claim 5 , wherein the prefetch engine is further configured to store at least a portion of the prefetch hint in the MS cache.
9. The device of claim 8 , wherein the prefetch engine is further configured to maintain a population of cache lines in the MS cache according to a priority established, at least in part, by the prefetch hint.
10. The device of claim 5 , wherein the prefetch engine is further configured to:
identify a full state of the MS cache;
determine that the prefetch data has a higher priority compared to at least a portion of cache lines stored in the MS cache;
evict at least a portion of the cache lines having a lower priority than the prefetch data; and
store the prefetch data in the MS cache.
11. The device of claim 1 , wherein the correlation with the read request includes a correlation with the read data.
12. The device of claim 5 , wherein the prefetch hint is retrieved from a model-specific register (MSR) configuration encoded in at least one MSR.
13. A computing system, comprising:
a processor-side device, further comprising;
a processor; and
a memory controller communicatively coupled to the processor;
a memory-side (MS) device, further comprising:
a nonvolatile memory (NVM) configured as system memory;
a MS cache communicatively coupled to the NVM and operable to store a cached subset of the NVM, the MS cache including volatile memory; and
a MS controller communicatively coupled to the NVM, to the MS cache, and to the memory controller, the MS controller including MS controller circuitry configured to;
retrieve read data from a memory address in the NVM to fill a read request from the memory controller;
check for a correlation between the read request and other data in the NVM;
retrieve prefetch data having the correlation with the read request; and
store the prefetch data in the MS cache.
14. The system of claim 13 , wherein the memory controller comprises circuitry configured to:
receive the read request from the processor, where the read request includes:
a thread identification (TID) of a hardware thread sending the read request; and
the memory address of the read data; and
send the read request to the MS controller.
15. The system of claim 14 , wherein the memory controller circuitry is further configured to:
determine a prefetch hint including the correlation between the read request and the prefetch data; and
include the prefetch hint in the read request.
16. The system of claim 15 , wherein, to determine the prefetch hint, the memory controller circuitry is further configured to receive the prefetch hint from the processor with the read request.
17. The system of claim 16 , wherein the processor is configured to receive the prefetch hint from software.
18. The system of claim 15 , wherein the memory controller further comprises a prefetch hint engine configured to determine the prefetch hint.
19. The system of claim 18 , wherein the prefetch hint engine is further configured to determine the prefetch hint from the TID.
20. The system of claim 19 , wherein the prefetch hint engine is further configured to determine the prefetch hint from a model-specific register (MSR) configuration of at least one pre-configured MSR that correlates the prefetch hint with the TID.
21. The system of claim 20 , wherein, to send the prefetch hint to the MS controller, the prefetch hint engine is further configured to provide the MSR configuration to the MS controller.
22. The system of claim 13 , wherein the processor includes a plurality of cores.
23. The system of claim 13 , wherein the processor is a plurality of processors.
24. A method of adaptively prefetching system memory data into a memory-side (MS) cache, comprising:
receiving, from a processor, a read request for read data in a nonvolatile memory (NVM) configured as system memory, where the read request includes:
a thread identification (TID) of a hardware thread sending the read request; and
a memory address of the read data associated with the read request;
determining, by a memory controller, a prefetch hint that includes an adaptive correlation between the read request and prefetch data;
determining a prefetch pattern in a MS controller from, at least in part, the prefetch hint;
retrieving from the NVM, by the MS controller according to the prefetch pattern, the prefetch data having the adaptive correlation to the read request; and
storing, by the MS controller, at least a portion of the prefetch data in a MS cache of volatile memory communicatively coupled to the NVM.
25. The method of claim 24 , wherein, in determining the prefetch hint, the method further comprises receiving, at the memory controller, the prefetch hint from the processor with the read request.
26. The method of claim 24 , further comprising determining the prefetch hint within the memory controller.