Scheduling multiple processing-in-memory (PIM) threads and non-PIM threads
Scheduling requests of multiple processing-in-memory threads and requests of multiple non-processing-in-memory threads is described. In accordance with the described techniques, a memory controller receives a plurality of processing-in-memory threads and a plurality of non-processing-in-memory threads from a host. The memory controller schedules an order of execution for requests of the plurality of processing-in-memory threads and requests of the plurality of non-processing-in-memory threads based on a priority associated with each of the requests and a current operating mode of the system. Requests are maintained in queues at the memory controller and are individually assigned a priority level based on time enqueued at the memory controller. Requests of a different mode than a current operating mode of the system are delayed for scheduling until at least one different mode request is escalated to a maximum priority value, at which point the memory controller initiates a system mode switch.
1. A system comprising:
memory hardware including a memory and a processing-in-memory component;
a host including at least one core; and
a memory controller configured to:
receive, from the host, a plurality of processing-in-memory threads and a plurality of non-processing-in-memory threads; and
schedule an order of execution for requests of the plurality of processing-in-memory threads and requests of the plurality of non-processing-in-memory threads based on a priority associated with each of the requests and a current operating mode of the system.
2. The system of claim 1 , wherein the current operating mode of the system comprises executing a first type of request and the order of execution prioritizes scheduling the first type of request in response to a second type of request not having a maximum priority level.
3. The system of claim 1 , wherein the current operating mode of the system comprises executing a first type of request and the order of execution prioritizes scheduling a second type of request in response to a second type of request having a maximum priority level.
4. The system of claim 3 , wherein the first type of request comprises a processing-in-memory request and the second type of request comprises a non-processing-in-memory request.
5. The system of claim 3 , wherein the first type of request comprises a non-processing-in-memory request and the second type of request comprises a processing-in-memory request.
6. The system of claim 1 , wherein the memory hardware is configured to execute requests of the plurality of processing-in-memory threads by causing the processing-in-memory component to execute one or more operations using data stored in the memory.
7. The system of claim 1 , wherein the host is configured to execute requests of the plurality of non-processing-in-memory threads by obtaining data stored in the memory and causing the at least one core to execute one or more operations using the data.
8. The system of claim 1 , wherein the order of execution comprises scheduling requests of the plurality of processing-in-memory threads in an order that is different from an order in which the requests of the plurality of processing-in-memory threads are received at the memory controller from the host.
9. The system of claim 1 , wherein the memory controller is associated with a channel in the memory and the channel in the memory is allocated into two or more pseudo-channels.
10. The system of claim 1 , wherein the memory controller is configured to schedule the order of execution by selecting, for each pseudo-channel of the memory, different types of requests from the plurality of processing-in-memory threads and different types of requests from the plurality of non-processing-in-memory threads using a first arbiter.
11. The system of claim 10 , wherein the first arbiter selects, for each pseudo-channel of the memory, one hit request, one miss request, and one conflict request from the plurality of processing-in-memory threads.
12. The system of claim 10 , wherein the first arbiter selects, for each pseudo-channel of the memory, one hit request, one miss request, and one conflict request from the plurality of non-processing-in-memory threads.
13. The system of claim 10 , wherein the memory controller is configured to schedule the order of execution by selecting, using a second arbiter and for each pseudo-channel of the memory, one request from the different types of requests from the plurality of processing-in-memory threads and the different types of requests from the plurality of non-processing-in-memory threads that were selected by the first arbiter.
14. The system of claim 13 , wherein the second arbiter is configured to select the one request for each pseudo-channel of the memory based on priority information associated with each of the different types of requests from the plurality of processing-in-memory threads and the different types of requests from the plurality of non-processing-in-memory threads that were selected by the first arbiter.
15. The system of claim 13 , wherein the memory controller is configured to schedule the order of execution using a third arbiter that defines an ordering of requests selected by the second arbiter.
16. A method comprising:
identifying, by a memory controller of a system, that the system is operating in a processing-in-memory mode;
receiving, by the memory controller from a host of the system, a plurality of processing-in-memory threads and a plurality of non-processing-in-memory threads; and
scheduling, by the memory controller, an order of execution for requests of the plurality of processing-in-memory threads and requests of the plurality of non-processing-in-memory threads by:
prioritizing requests of the plurality of processing-in-memory threads responsive to identifying that requests of the plurality of non-processing-in-memory threads do not satisfy a maximum priority stall threshold; and
prioritizing requests of the plurality of non-processing-in-memory threads responsive to identifying that at least one of the requests of the plurality of non-processing-in-memory threads satisfies the maximum priority stall threshold.
17. The method of claim 16 , wherein prioritizing the requests of the plurality of non-processing-in-memory threads is further performed responsive to identifying that the requests of the plurality of processing-in-memory threads do not satisfy the maximum priority stall threshold.
18. The method of claim 16 , further comprising incrementing stall counters for requests enqueued at a request queue of the memory controller and updating priority levels of the requests enqueued at the request queue during the scheduling the order of execution.
19. A device comprising:
a memory controller configured to:
identify that the device is operating in a non-processing-in-memory mode;
receive a plurality of processing-in-memory threads and a plurality of non-processing-in-memory threads; and
schedule an order of execution for requests of the plurality of processing-in-memory threads and requests of the plurality of non-processing-in-memory threads by:
prioritizing requests of the plurality of non-processing-in-memory threads responsive to identifying that requests of the plurality of processing-in-memory threads do not satisfy a maximum priority stall threshold; and
prioritizing requests of the plurality of processing-in-memory threads responsive to identifying that at least one of the requests of the plurality of processing-in-memory threads satisfies the maximum priority stall threshold.
20. The device of claim 19 , wherein the memory controller prioritizes the requests of the plurality of processing-in-memory threads responsive to identifying that the requests of the plurality of non-processing-in-memory threads do not satisfy the maximum priority stall threshold.