Speculative cache invalidation for processing-in-memory instructions
Speculative cache invalidation techniques for processing-in-memory instructions are described. In one example, a system includes a cache system including a plurality of cache levels and a cache coherence controller. The cache coherence controller is configured to perform a cache directory lookup using a cache directory. The cache directory lookup is configured to indicate whether data associated with a memory address specified by a processing-in-memory request is valid in memory. The system employs speculative evaluation logic to identify whether the data associated with the processing-in-memory request is stored in the cache system before the processing-in-memory request is transmitted to the cache coherence controller. If the data is stored in the cache system, the cache system locally invalidates or flushes the data to avoid stalling the processing-in-memory request during a cache directory lookup.
1 . A system comprising:
a cache controller including a cache directory;
a processing-in-memory component; and
a cache system including a plurality of cache levels, the cache system configured to:
ensure validity, using a tag table maintained in one of the plurality of cache levels, of one or more data elements involved in executing a processing-in-memory request before the processing-in-memory request is dispatched to the cache controller or the processing-in-memory component; and
cause the cache controller to skip performing a lookup for the processing-in-memory request using the cache directory responsive to ensuring validity of the one or more data elements using the tag table.
2 . The system of claim 1 , wherein the cache system is configured to ensure the validity of the one or more data elements by invalidating the one or more data elements from the cache system.
3 . The system of claim 1 , wherein the cache system is configured to ensure the validity of the one or more data elements by flushing the one or more data elements from the cache system to memory accessible by the processing-in-memory component.
4 . The system of claim 1 , wherein the cache system ensures the validity of the one or more data elements involved in executing the processing-in-memory request by comparing a memory address for the one or more data elements to the tag table, wherein the tag table includes metadata describing data stored in the cache system.
5 . The system of claim 1 , wherein the processing-in-memory request includes a speculation bit that causes the cache controller to not transmit a cache request to flush the one or more data elements from the cache system upon receiving the processing-in-memory request.
6 . The system of claim 1 , wherein the cache directory includes a plurality of cache directory entries that indicate memory addresses that are maintained in the cache system.
7 . The system of claim 6 , wherein the plurality of cache directory entries specify, respectively, a location of respective memory addresses in a plurality of cache levels of the cache system and a status of the respective memory address.
8 . The system of claim 1 , wherein the cache system is configured to allow transmission of the processing-in-memory request to the cache controller in response to determining that the one or more data elements are valid in memory for use in executing the processing-in-memory request by the processing-in-memory component.
9 . The system of claim 1 , wherein the cache system is configured to ensure validity of one or more additional data elements involved in executing an additional processing-in-memory request before the additional processing-in-memory request is issued by a processing unit.
10 . The system of claim 9 , wherein the cache system is configured to select the one or more additional data elements involved in executing the additional processing-in-memory request based on spatial locality relative to the one or more data elements involved in executing the processing-in-memory request.
11 . A device comprising:
a cache system comprising a plurality of cache levels and configured to:
receive a processing-in-memory request to be executed by a processing-in-memory component;
ensure validity, using a tag table maintained in one of the plurality of cache levels, of one or more data elements involved in executing the processing-in-memory request before the processing-in-memory request is transmitted to a memory controller for scheduling execution at the processing-in-memory component; and
responsive to ensuring validity of the one or more data elements using the tag table maintained in the one of the plurality of cache levels, cause a cache controller coupled to the device to skip performing a lookup for the processing-in-memory request using a cache directory.
12 . The device of claim 11 , wherein the cache system ensures the validity of the one or more data elements involved in executing the processing-in-memory request by invalidating the one or more data elements from the cache system.
13 . The device of claim 11 , wherein the cache system ensures the validity of the one or more data elements involved in executing the processing-in-memory request by flushing the one or more data elements from the cache system to memory accessible by the processing-in-memory component.
14 . The device of claim 11 , wherein the cache system ensures the validity of the one or more data elements involved in executing the processing-in-memory request by comparing a memory address for the one or more data elements to the tag table, wherein the tag table includes metadata describing data stored in the cache system.
15 . The device of claim 11 , wherein transmitting the processing-in-memory request comprises routing the processing-in-memory request from a processing unit to the memory controller via the cache controller, and wherein the processing-in-memory request includes a speculation bit that causes the cache controller to not perform the lookup for the processing-in-memory request using the cache directory.
16 . The device of claim 15 , wherein the cache controller comprises a cache directory that includes a plurality of cache directory entries that indicate memory addresses that are maintained in the cache system.
17 . The device of claim 11 , wherein the cache system is further configured to ensure validity of one or more additional data elements involved in executing an additional processing-in-memory request before the additional processing-in-memory request is issued by a processing unit.
18 . The device of claim 17 , wherein the one or more additional data elements involved in executing the additional processing-in-memory request are selected based on spatial locality relative to the one or more data elements involved in executing the processing-in-memory request.
19 . A method comprising:
detecting, by a cache system comprising a plurality of cache levels, issuance of a processing-in-memory request from a processing unit core to be executed by a processing-in-memory component;
ensuring validity, by the cache system using a tag table maintained in one of the plurality of cache levels, of one or more data elements involved in executing the processing-in-memory request by invalidating or flushing the one or more data elements to memory that is accessible by the processing-in-memory component before the processing-in-memory request is dispatched to a cache controller or a memory controller of a device implementing the processing-in-memory component; and
responsive to ensuring validity of the one or more data elements using the tag table maintained in the one of the plurality of cache levels, causing the cache controller to skip performing a lookup for the processing-in-memory request using a cache directory.
20 . The method of claim 19 , wherein the plurality of cache levels are arranged in a cache hierarchy from a highest-level cache to a lowest-level cache, wherein the tag table is maintained in the lowest-level cache.