Approach for enabling concurrent execution of host memory commands and near-memory processing commands
An approach allows concurrent execution of near-memory processing commands, referred to herein as “PIM commands,” and host memory commands. A memory controller determines and issues a plurality of register-only PIM commands that do not reference memory with host memory commands to allow concurrent execution of the register-only PIM commands and the host memory commands. The approach allows concurrent execution of register-only PIM commands and host memory commands without interference, even when the register-only PIM commands and the host memory commands are interleaved, and even for the same memory module, which improves resource utilization and performance. Further improvement of resource utilization and performance is achieved by extending a register-only phase by reordering register-only PIM commands before non-register-only PIM commands, subject to dependency constraints, and using shadow row buffers to provide local working copies of data from memory to near-memory compute elements.
1. A memory command processing element configured to:
determine a plurality of near-memory processing commands that do not reference memory, and
cause issuance of the plurality of near-memory processing commands that do not reference memory and one or more host memory commands that do reference memory to allow the plurality of near-memory processing commands that do not reference memory and the one or more host memory commands that do reference memory to be processed concurrently.
2. The memory command processing element of claim 1 , wherein the one or more host memory commands that do reference memory are interleaved with the plurality of near-memory processing commands that do not reference memory.
3. The memory command processing element of claim 1 , wherein the plurality of near-memory processing commands that do not reference memory includes at least a specified number of near-memory processing commands that do not reference memory without any near-memory processing commands that reference memory.
4. The memory command processing element of claim 1 , wherein:
the plurality of near-memory processing commands that do not reference memory includes one or more near-memory processing commands that were enqueued after a near-memory processing command that references memory, and
there are no data dependencies via registers or memory between the near-memory processing command that references memory and the one or more near-memory processing commands that were enqueued after the near-memory processing command that references memory.
5. The memory command processing element of claim 1 , further configured to:
determine a second plurality of commands that includes one or more near-memory processing commands that do not reference memory, one or more near-memory processing commands that reference memory, and no host memory commands, and
cause the second plurality of commands to be issued.
6. The memory command processing element of claim 5 , wherein the second plurality of commands is identified by one or more of a first near-memory processing command in the second plurality of commands or a separate command received by the memory command processing element.
7. The memory command processing element of claim 1 , further configured to:
determine a second plurality of near-memory processing commands that reference a particular row in a memory, and
cause a copy of data stored in the particular row in the memory to be stored in a local buffer of a near-memory processing element and modify the second plurality of near-memory processing commands to reference the local buffer of the near-memory processing element instead of the particular row in the memory.
8. The memory command processing element of claim 7 , wherein the copy of data stored in the particular row in the memory that is stored in the local buffer of the near-memory processing element is data stored at column addresses specified by the second plurality of near-memory processing commands or all of data in the particular row in the memory.
9. The memory command processing element of claim 7 , further configured to prevent the copy of data stored in the local buffer of the near-memory processing element from being updated.
10. The memory command processing element of claim 7 , further configured to cause a modified copy of the copy of data from stored in the local buffer of the near-memory processing element to be:
stored in the local buffer of the near-memory processing element and the data stored in the particular row in the memory designated as non-accessible, or
stored in the particular row in the memory.
11. The memory command processing element of claim 1 , further configured to:
determine two or more near-memory processing commands that reference different rows in a memory, and
cause a copy of data stored at the different rows in the memory to be stored in a local buffer of a near-memory processing element and modify the two or more near-memory processing commands to reference the local buffer of the near-memory processing element instead of the memory.
12. The memory command processing element of claim 11 , wherein the copy of data stored at the different rows in the memory is stored in a single row in the local buffer of the near-memory processing element or multiple rows in the local buffer of the near-memory processing element.
13. The memory command processing element of claim 1 , wherein the memory command processing element is one or more of a memory controller, a cache, a queue, or a buffer.
14. A near-memory processing element comprising local storage;
wherein the near-memory processing element is configured to:
store, in the local storage, data from a plurality of memory locations referenced by a plurality of near-memory processing commands, and
process the plurality of near-memory processing commands using the data from the plurality of memory locations stored in the local storage to allow one or more host memory commands that reference one or more memory locations from the plurality of memory locations to be processed concurrently.
15. The near-memory processing element of claim 14 , wherein the plurality of memory locations referenced by the plurality of near-memory processing commands are in a single row of a memory.
16. The near-memory processing element of claim 15 , further configured to cause a modified version of the data stored in the local storage to be stored to the memory.
17. A memory controller configured to:
determine a plurality of near-memory processing commands that do not reference memory, and
issue the plurality of near-memory processing commands that do not reference memory and one or more host memory commands that do reference memory to allow the plurality of near-memory processing commands that do not reference memory and the one or more host memory commands that do reference memory to be processed concurrently.
18. The memory controller of claim 17 , wherein the one or more host memory commands that do reference memory are interleaved with the plurality of near-memory processing commands that do not reference memory.
19. The memory controller of claim 17 , wherein the plurality of near-memory processing commands that do not reference memory includes at least a specified number of near-memory processing commands that do not reference memory without any near-memory processing commands that reference memory.
20. The memory controller of claim 17 , wherein:
the plurality of near-memory processing commands that do not reference memory includes one or more near-memory processing commands that were enqueued after a near-memory processing command that references memory, and
there are no data dependencies via registers or memory between the near-memory processing command that references memory and the one or more near-memory processing commands that were enqueued after the near-memory processing command that references memory.