Bank-level parallelism for processing in memory
In accordance with the described techniques for bank-level parallelism for processing in memory, a plurality of commands are received for execution by a processing in memory component embedded in a memory. The memory includes a first bank and a second bank. The plurality of commands include a first stream of commands which cause the processing in memory component to perform operations that access the first bank and a second stream of commands which cause the processing in memory component to perform operations that access the second bank. A next row of the first bank that is to be accessed by the processing in memory component is identified. Further, a precharge command is scheduled to close a first row of the first bank and an activate command is scheduled to open the next row of the first bank in parallel with execution of the second stream of commands.
1. A computing device comprising:
a memory that includes a first bank and a second bank;
a processing in memory circuit; and
a memory controller to:
receive a plurality of commands including a first stream of one or more commands that causes the processing in memory circuit to perform operations that access a first row of the first bank, and a second stream of one or more commands that causes the processing in memory circuit to perform operations that access the second bank; and
schedule a precharge command to close the first row of the first bank and an activate command to open a next row of the first bank that is to be accessed by the processing in memory component after the first stream of one or more commands, the precharge command and the activate command scheduled to execute in parallel with execution of the second stream of one or more commands.
2. The computing device of claim 1 , wherein the plurality of commands include an identification command that identifies the next row of the first bank, and the memory controller is configured to schedule the precharge command and the activate command based on the identification command.
3. The computing device of claim 1 , wherein the memory is organized into a plurality of blocks, each block including a bank assigned to a first group, a bank assigned to a second group, and a respective processing in memory circuit shared among the banks included in a respective block.
4. The computing device of claim 3 , wherein the first bank is assigned to the first group, the second bank is assigned to the second group, and the processing in memory circuit is shared among the first bank and the second bank.
5. The computing device of claim 3 , wherein the activate command causes banks assigned to the first group to open the next row in parallel, and the second stream of one or more commands causes multiple processing in memory circuits to perform operations that access banks assigned to the second group in parallel.
6. The computing device of claim 1 , wherein the memory controller is further configured to schedule the activate command to open the next row based on an inspection of rows accessed by the plurality of commands.
7. The computing device of claim 1 , wherein the memory controller is further configured to predict, using a prediction policy, the next row of the first bank to be accessed by the processing in memory circuit based on access patterns of previously issued processing in memory commands, and schedule the activate command to open the next row based on the prediction.
8. The computing device of claim 7 , wherein the memory controller is further configured to update the prediction policy based on whether a row of the first bank that is opened based on the prediction is a row of the first bank that is accessed next by the processing in memory circuit.
9. The computing device of claim 7 , wherein the memory controller is further configured to:
receive one or more hints indicative of at least one of: an amount of data that is to be processed by the processing in memory circuit in executing the plurality of commands or a number of data structures involved in the plurality of commands; and
predict the next row of the first bank to be accessed by the processing in memory circuit based on the one or more hints.
10. The computing device of claim 1 , wherein the precharge command is scheduled to close the first row of the first bank in parallel with execution of a first set of the second stream of one or more commands having a combined execution time that is equal to or greater than an overhead time for closing a row of the memory.
11. The computing device of claim 10 , wherein the activate command is scheduled after the first set of the second stream of one or more commands to open the next row in parallel with execution of a second set of the second stream of one or more commands by the processing in memory circuit.
12. A system comprising;
a memory that includes a first group of banks and a second group of banks;
multiple processing in memory circuits; and
a processor to:
receive a first stream of one or more commands that causes the multiple processing in memory circuits to perform operations that access the first group of banks in parallel, and a second stream of one or more commands that causes the multiple processing in memory circuits to perform operations that access the second group of banks in parallel; and
schedule an activate command that causes the first group of banks to open a next row of the first group of banks that is to be accessed by the multiple processing in memory circuits after the first stream of commands in parallel, the activate command scheduled to execute in parallel with execution of the second stream of one or more commands.
13. The system of claim 12 , wherein the processor is further configured to receive an identification command that identifies the next row of the first group of banks, and schedule the activate command based on the identification command.
14. The system of claim 12 , wherein the memory is organized into a plurality of blocks, each including a bank assigned to the first group, a bank assigned to the second group, and a processing in memory circuit shared among the banks in a respective block.
15. The system of claim 12 , wherein the processor is further configured to schedule a precharge command to close a first row of the first group of banks that is accessed by the first stream of one or more commands, the precharge command scheduled to execute in parallel with execution of the second stream of one or more commands.
16. The system of claim 12 , wherein the processor is configured to:
predict, using a prediction policy, the next row of the first group of banks to be accessed by the multiple processing in memory circuits; and
update the prediction policy based on whether a row of the first group of banks that is opened based on the prediction is a row of the first group of banks that is accessed next by the multiple processing in memory circuits.
17. A method comprising:
receiving a first stream of commands that causes a processing in memory circuit to access a first bank of a memory, a second stream of commands that causes the processing in memory circuit to access a second bank of the memory, and an identification command ordered before the second stream of commands that identifies a next row of the first bank to be accessed by the processing in memory circuit after the second stream of one or more commands; and
scheduling an activate command to open the next row of the first bank in parallel with execution of the second stream of one or more commands.
18. The method of claim 17 , further comprising scheduling a precharge command to close a first row of the first bank that is accessed by the first stream of one or more commands in parallel with execution of the second stream of one or more commands.
19. The method of claim 18 , further comprising scheduling one or more commands of the second stream of one or more commands after the precharge command and before the activate command, the one or more commands having a combined execution time that is greater than or equal to an overhead time for closing a row of the memory.
20. The method of claim 17 , wherein the identification command identifies the next row of the first bank without causing data to be read from or written to the next row.