Preemptive flushing of processing-in-memory data structures
Preemptive flushing of data involved in executing a processing-in-memory command, from a cache system to main memory that is accessible by a processing-in-memory component, is described. In one example, a system includes an asynchronous flush controller that receives an indication of a subsequent processing-in-memory command to be executed as part of performing a computational task. While earlier commands of the computational task are executed, the asynchronous flush controller evicts or invalidates data elements involved in executing the subsequent processing-in-memory command from the cache system, such that the processing-in-memory command can proceed without stalling.
1 . A system comprising:
a cache system including a plurality of cache levels; and
a cache controller configured to:
identify that a processing-in-memory command involves a plurality of data elements stored at an address range in the cache system; and
responsive to the address range satisfying a threshold size, evict a portion of the plurality of data elements from the cache system prior to issuance of the processing-in-memory command, the evicted portion of the plurality of data elements excluding some of the plurality of data elements.
2 . The system of claim 1 , wherein the processing-in-memory command is associated with a computational task, and wherein the cache controller is configured to identify that the processing-in-memory command involves the plurality of data elements stored at the address range in the cache system based on code for the computational task including a non-processing-in-memory command that modifies at least one of the plurality of data elements prior to issuance of the processing-in-memory command.
3 . The system of claim 1 , wherein the cache controller is configured to evict the portion of the plurality of data elements from the cache system by transmitting a clean command to the cache system that specifies a portion of the address range including the portion of the plurality of data elements and an instruction to flush the portion of the plurality of data elements to memory of the system or invalidate the portion of the plurality of data elements in the cache system.
4 . The system of claim 3 , wherein the processing-in-memory command is associated with a computational task, and wherein the cache controller generates the clean command by performing a cache directory lookup for the portion of the plurality of data elements during execution of one or more different commands involved in performing the computational task.
5 . The system of claim 4 , wherein the cache controller generates the portion of the address range of the clean command based on one or more cache hit responses to the cache directory lookup.
6 . The system of claim 4 , wherein the cache controller performs the cache directory lookup by skipping a beginning portion of the plurality of data elements based on the address range in the cache system satisfying the threshold size.
7 . The system of claim 6 , wherein the threshold size for the address range in the cache system is based on a storage size of the cache system.
8 . The system of claim 4 , wherein the cache controller performs the cache directory lookup by skipping a threshold number of the plurality of data elements based on the cache directory lookup returning a number of successive cache misses.
9 . The system of claim 3 , wherein the processing-in-memory command is associated with a computational task, and wherein the cache controller transmits the clean command during execution of one or more different commands involved in performing the computational task.
10 . A device comprising:
a cache controller configured to:
receive an indication that a processing-in-memory command to be subsequently executed involves a plurality of data elements;
perform a cache directory lookup to determine whether the plurality of data elements are stored at an address range in a cache system; and
responsive to the address range satisfying a threshold size, evict a portion of the plurality of data elements from the cache system prior to issuance of the processing-in-memory command, the evicted portion of the plurality of data elements excluding some of the plurality of data elements.
11 . The device of claim 10 , wherein the cache controller evicts the portion of the plurality of data elements from the cache system by transmitting a clean command to the cache system that specifies a portion of the address range including the plurality of data elements and an instruction to flush the portion of the plurality of data elements to memory or invalidate the portion of the plurality of data elements in the cache system.
12 . The device of claim 11 , wherein the cache controller performs the cache directory lookup by skipping a beginning portion of the plurality of data elements included in the address range in response to the address range satisfying the threshold size.
13 . The device of claim 12 , wherein the threshold size for the address range is based on a storage size of the cache system.
14 . The device of claim 11 , wherein the processing-in-memory command is associated with a computational task, and wherein the cache controller generates the clean command by performing a cache directory lookup for the portion of the plurality of data elements included in the address range during execution of one or more different commands involved in performing the computational task.
15 . The device of claim 14 , wherein the cache controller generates the portion of the address range specified by the clean command based on one or more cache hit responses to the cache directory lookup.
16 . The device of claim 11 , wherein the cache controller performs the cache directory lookup by skipping a threshold number of the plurality of data elements based on the cache directory lookup returning a number of successive cache misses.
17 . The device of claim 11 , wherein the processing-in-memory command is associated with a computational task, and wherein the cache controller transmits the clean command during execution of one or more different commands involved in performing the computational task.
18 . The device of claim 11 , wherein the processing-in-memory command is associated with a computational task, and wherein the indication that the processing-in-memory command involves the plurality of data elements stored in the cache system is generated based on code for the computational task including a non-processing-in-memory command that modifies at least one of the plurality of data elements prior to issuance of the processing-in-memory command.
19 . A method comprising:
executing, by a core of a processing unit, a plurality of commands as part of performing a computational task;
identifying that the plurality of commands include a processing-in-memory instruction that accesses a plurality of data elements stored at an address range in a cache system during performance of the computational task;
identifying that the address range satisfies a threshold size;
responsive to the address range satisfying the threshold size, and while the core of the processing unit is executing a portion of the plurality of commands that occur prior to the processing-in-memory instruction, evicting a portion of the plurality of data elements from the cache system, the portion of the plurality of data elements excluding at least one of the plurality of data elements; and
transmitting, by the core of the processing unit, the processing-in-memory instruction to a processing-in-memory component after the portion of the plurality of data elements is evicted from the cache system.
20 . The method of claim 19 , wherein evicting the portion of the plurality of data elements is performed without evicting all of the plurality of data elements from the cache system.