Adaptive scheduling of memory and processing-in-memory requests
Adaptive scheduling of memory requests and processing-in-memory requests is described. In accordance with the described techniques, a memory controller receives a plurality of processing-in-memory requests and a plurality of non-processing-in-memory requests from a host. The memory controller schedules an order of execution for the plurality of processing-in-memory requests and the plurality of non-processing-in-memory requests based at least in part on a processing-in-memory request stall threshold and a non-processing-in-memory request stall threshold. In response to a system switching (e.g., from executing processing-in-memory requests to executing non-processing-in-memory requests or from executing non-processing-in-memory requests to executing processing-in-memory requests), the memory controller modifies the processing-in-memory request stall threshold and the non-processing-in-memory request stall threshold. The memory controller continues scheduling an order of execution for subsequent requests received from the host using the modified stall thresholds.
1. A system comprising:
a memory module including a memory and a processing-in-memory component;
a host including at least one core; and
a memory controller configured to:
receive a plurality of processing-in-memory requests and a plurality of non-processing-in-memory requests from the host;
schedule an order of execution for the plurality of processing-in-memory requests and the plurality of non-processing-in-memory requests based at least in part on a processing-in-memory request stall threshold and a non-processing-in-memory request stall threshold; and
modifying the processing-in-memory request stall threshold and the non-processing-in-memory request stall threshold in response to the system switching between executing processing-in-memory requests and executing non-processing-in-memory requests.
2. The system of claim 1 , wherein executing the processing-in-memory requests comprises causing the processing-in-memory component to execute one or more operations of a transaction using data stored in the memory.
3. The system of claim 1 , wherein executing the non-processing-in-memory requests comprises transmitting data stored in the memory from the memory module to the host and executing, by the at least one core, one or more operations of a transaction using the data.
4. The system of claim 1 , wherein the memory controller is associated with a channel in the memory and the channel in the memory is allocated into two or more pseudo-channels.
5. The system of claim 4 , wherein the memory controller is configured to schedule the order of execution by selecting, for each of the two or more pseudo-channels, different types of the plurality of processing-in-memory requests and different types of the non-processing-in-memory requests using a first arbiter.
6. The system of claim 5 , wherein the first arbiter selects, for each of the two or more pseudo-channels, one hit request, one miss request, and one conflict request from the plurality of processing-in-memory requests.
7. The system of claim 5 , wherein the first arbiter selects, for each of the two or more pseudo-channels, one hit request, one miss request, and one conflict request from the plurality of non-processing-in-memory requests.
8. The system of claim 5 , wherein the memory controller is configured to schedule the order of execution by selecting, using a second arbiter and for each of the two or more pseudo-channels, one request from the different types of the plurality of processing-in-memory requests and the different types of the non-processing-in-memory requests that were selected by the first arbiter.
9. The system of claim 8 , wherein the second arbiter is configured to select the one request for each of the two or more pseudo-channels based on priority information associated with each of the different types of the plurality of processing-in-memory requests and the different types of the non-processing-in-memory requests that were selected by the first arbiter.
10. The system of claim 8 , wherein the second arbiter is configured to select the one request for each of the two or more pseudo-channels based on a time at which each of the different types of the plurality of processing-in-memory requests and the different types of the non-processing-in-memory requests that were selected by the first arbiter is stalled at the memory controller.
11. The system of claim 8 , wherein the second arbiter is configured to select the one request for each of the two or more pseudo-channels based on an estimated cost of:
switching the system from executing processing-in-memory requests to executing non-processing-in-memory requests; or
switching the system from executing non-processing-in-memory requests to executing processing-in-memory requests.
12. The system of claim 8 , wherein the memory controller is configured to schedule the order of execution using a third arbiter that defines an ordering of requests selected by the second arbiter.
13. The system of claim 1 , wherein the memory controller is further configured to predict, while the system is executing processing-in-memory requests, a first cycle for switching to executing non-processing-in-memory requests based on historical data describing a number of hits per row and a measured number of open rows in the memory while the system was previously executing non-processing-in-memory requests.
14. The system of claim 13 , wherein the memory controller is further configured to predict, while the system is executing non-processing-in-memory requests, a second cycle for switching to executing processing-in-memory requests based on historical data describing a measured number of hits per row in the memory while the system was previously executing processing-in-memory requests.
15. The system of claim 14 , wherein the memory controller is further configured to:
switch the system from executing processing-in-memory requests to executing non-processing-in-memory request at the first cycle; and
switch the system from executing non-processing-in-memory requests to executing processing-in-memory requests at the second cycle.
16. The system of claim 15 , wherein the memory controller is further configured to compute a difference between a time at which a first request executed after the system switching was stalled at the memory controller and a corresponding one of the processing-in-memory request stall threshold or the non-processing-in-memory request stall threshold.
17. The system of claim 16 , wherein the first request executed after the system switching is a processing-in-memory request and modifying the processing-in-memory request stall threshold and the non-processing-in-memory request stall threshold comprises:
generating a modified processing-in-memory request stall threshold by adding the difference to the processing-in-memory request stall threshold; and
generating a modified non-processing-in-memory request stall threshold by subtracting the difference from the non-processing-in-memory request stall threshold.
18. The system of claim 16 , wherein the first request executed after the system switching is a non-processing-in-memory request and modifying the processing-in-memory request stall threshold and the non-processing-in-memory request stall threshold comprises:
generating a modified processing-in-memory request stall threshold by subtracting the difference from the processing-in-memory request stall threshold; and
generating a modified non-processing-in-memory request stall threshold by adding the difference to the non-processing-in-memory request stall threshold.
19. A method comprising:
switching, by a memory controller of a system, operation of the system from executing non-processing-in-memory requests to executing processing-in-memory requests; and
modifying, by the memory controller, a processing-in-memory request stall threshold for the system and a non-processing-in-memory request stall threshold for the system in response to the switching by:
computing a difference between the processing-in-memory request stall threshold and a time at which a first processing-in-memory request executed after the switching was stalled at the memory controller;
generating a modified processing-in-memory request stall threshold by adding the difference to the processing-in-memory request stall threshold; and
generating a modified non-processing-in-memory request stall threshold by subtracting the difference from the non-processing-in-memory request stall threshold.
20. A method comprising:
switching, by a memory controller of a system, operation of the system from executing processing-in-memory requests to executing non-processing-in-memory requests; and
modifying, by the memory controller, a processing-in-memory request stall threshold for the system and a non-processing-in-memory request stall threshold for the system in response to the switching by:
computing a difference between the non-processing-in-memory request stall threshold and a time at which a first non-processing-in-memory request executed after the switching was stalled at the memory controller;
generating a modified non-processing-in-memory request stall threshold by adding the difference to the non-processing-in-memory request stall threshold; and
generating a modified processing-in-memory request stall threshold by subtracting the difference from the processing-in-memory request stall threshold.