Memory-aware pre-fetching and cache bypassing systems and methods
Systems, apparatuses, and methods related to memory management are described. For example, these may include a first memory level including memory pages in a memory array, a second memory level including a cache, a pre-fetch buffer, or both, and a memory controller that determines state information associated with a memory page in the memory array targeted by a memory access request. The state information may include a first parameter indicative of a current activation state of the memory page and a second parameter indicative of statistical likelihood (e.g., confidence) that a subsequent memory access request will target the memory page. The memory controller may disable storage of data associated with the memory page in the second memory level when the first parameter associated with the memory page indicates that the memory page is activated and the second parameter associated with the memory page is greater than or equal to a threshold.
1. An apparatus comprising:
a first hierarchical memory level comprising a memory array;
a second hierarchical memory level comprising a cache, a pre-fetch buffer, or both; and
a memory controller configured to be communicatively coupled to a memory bus, wherein the memory controller is configured to:
determine first state information associated with a memory page in the memory array targeted by a first memory access request received via the memory bus, wherein the first state information comprises:
a first parameter indicative of a current activation state of the memory page; and
a second parameter indicative of statistical likelihood that a subsequent memory access request will target the memory page, wherein the second parameter is determined based on one or more previously determined counter values each indicating a number of consecutive memory access requests previously targeting the memory page; and
disable storage of data associated with the memory page in the second hierarchical memory level when the first parameter in the first state information associated with the memory page indicates that the memory page is an activated memory page and the second parameter in the first state information associated with the memory page is greater than or equal to a first threshold.
2. The apparatus of claim 1 , wherein the memory controller is configured to instruct the apparatus to store a copy of the data associated with the memory page in the second hierarchical memory level when the first parameter in the first state information associated with the memory page indicates that the memory page is the activated memory page and the second parameter in the first state information associated with the memory page is less than the first threshold.
3. The apparatus of claim 1 , wherein, when the first parameter in the first state information associated with the memory page indicates that the memory page is not the activated memory page, the memory controller is configured to:
determine second state information associated with the activated memory page in the memory array, wherein the second state information comprises a third parameter indicative of statistical likelihood that the subsequent memory access request will target the activated memory page;
disable storage of the data associated with the memory page in the second hierarchical memory level when the third parameter in the second state information associated with the activated memory page is greater than a second threshold; and
instruct the apparatus to store a copy of the data associated with the memory page in the second hierarchical memory level when the third parameter in the second state information associated with the activated memory page is less than or equal to the second threshold.
4. The apparatus of claim 1 , wherein the memory controller is configured to enable storage of a copy of the data associated with the memory page in the second hierarchical memory level when first parameter in the first state information associated with the memory page indicates that the memory page is a deactivated memory page.
5. The apparatus of claim 1 , wherein, when the memory page is transitioned from a deactivated state to an activated state, the memory controller is configured to:
update the first parameter in the first state information associated with the memory page to indicate that the memory page is in the activated state;
reset the counter value; and
increment the counter value when a second memory access request to be fulfilled directly after the first memory access request targets the memory page.
6. The apparatus of claim 5 , wherein, when the memory page is subsequently transitioned from the activated state to the deactivated state, the memory controller is configured to:
update the first parameter in the first state information associated with the memory page to indicate that the memory page is in the deactivated state; and
update a value of the second parameter in the first state information associated with the memory page based at least in part on the counter value resulting when the memory page is transitioned to the deactivated state.
7. The apparatus of claim 5 , wherein the memory controller is configured to:
retrieve a first value of the second parameter in the first state information associated with the memory page when the memory page is transitioned from the deactivated state to the activated state; and
update the second parameter in the first state information associated with the memory page by overwriting the first value of the second parameter with a second value determined based at least in part on an average of the first value retrieved when the memory page is initially transitioned to the activated state and the counter value resulting when the memory page is subsequently transitioned to the deactivated state.
8. The apparatus of claim 1 , wherein, when the first memory access request comprises a write memory access request that demands that the data be stored at least in the memory array, the memory controller is configured to:
disable storage of the data in the second hierarchical memory level at least in part by enabling a cache bypass;
instruct the apparatus to store a copy of the data in the second hierarchical memory level when the cache bypass is not enabled; and
instruct the apparatus to store the data at a memory address in the memory page demanded by the write memory access request without storing the copy of the data in the second hierarchical memory level when the cache bypass is enabled.
9. The apparatus of claim 1 , comprising processing circuitry communicatively coupled to the memory controller, wherein, when the first memory access request comprises a read memory access request that demands return of the data associated with the memory page, the memory controller is configured to:
disable storage of the data in the second hierarchical memory level at least in part by enabling a cache bypass;
instruct the memory array to output the data demanded by the read memory access request from the memory page when the data is not currently stored in the second hierarchical memory level;
instruct the apparatus to store a copy of the data output from the memory array in the second hierarchical memory level when the cache bypass is not enabled; and
instruct the apparatus to return the data output from the memory array to the processing circuitry without storing the copy of the data in the second hierarchical memory level when the cache bypass is enabled.
10. The apparatus of claim 1 , wherein:
the first memory access request comprises a pre-fetch memory access request that targets pre-fetching of the data from a memory address in the memory page; and
the memory controller is configured to disable storage of the data in the second hierarchical memory level at least in part by blocking fulfillment of the pre-fetch memory access request.
11. A tangible, non-transitory, computer-readable medium storing instructions executable by control circuitry of a computing system, wherein the instructions comprise instructions to:
determine, using the control circuitry, a memory address in a memory array to which processing circuitry in the computing system is demanding access;
instruct, using the control circuitry, the computing system to provide access to a demanded memory page in the memory array comprising the memory address demanded by the processing circuitry;
determine, using the control circuitry, a current activation state of the demanded memory page comprising the memory address demanded by the processing circuitry; and
in response to determining that the current activation state of the demanded memory page is an activated state:
predict, using the control circuitry, a first statistical likelihood that a next memory address to be targeted directly after the memory address demanded by the processing circuitry will be in the demanded memory page, wherein the first statistical likelihood is determined based on one or more previously determined counter values each indicating a number of consecutive memory access requests previously targeting the memory page; and
enable, using the control circuitry, a cache bypass to disable caching of a data block associated with memory address demanded by the processing circuitry in response to determining that the first statistical likelihood that the next memory address to be targeted directly after the memory address demanded by the processing circuitry will be in the demanded memory page is greater than a first cache threshold.
12. The tangible, non-transitory, computer-readable medium of claim 11 , comprising instructions to disable, using the control circuitry, the cache bypass to enable caching of the data block associated with memory address demanded by the processing circuitry in response to determining that the current activation state of the demanded memory page is the activated state and the first statistical likelihood that the next memory address to be targeted directly after the memory address demanded by the processing circuitry will be in the demanded memory page is less than or equal to the first cache threshold.
13. The tangible, non-transitory, computer-readable medium of claim 11 , comprising instructions to, in response to determining that the current activation state of the demanded memory page is not the activated state:
predict, using the control circuitry, a second statistical likelihood that the next memory address to be targeted directly after the memory address demanded by the processing circuitry will be in a currently activated memory page in the memory array; and
enable, using the control circuitry, the cache bypass to disable caching of the data block associated with the memory address demanded by the processing circuitry in response to determining that the second statistical likelihood that the next memory address to be targeted directly after the memory address demanded by the processing circuitry will be in the currently activated memory page is not less than a second cache threshold.
14. The tangible, non-transitory, computer-readable medium of claim 13 , comprising instructions to disable, using the control circuitry, the cache bypass to enable caching of the data block associated with the memory address demanded by the processing circuitry in response to determining that:
the second statistical likelihood that the next memory address to be targeted directly after the memory address demanded by the processing circuitry will be in the currently activated memory page is less than the second cache threshold; or
the current activation state of the demanded memory page is not the activated state.