Method and mechanism for cache compaction and bandwidth reduction
A method and mechanism are managing caches. A cache is configured to store blocks of data based upon predictions of future accesses. Each block is partitioned into sub-blocks, and if it is predicted a given sub-block is unlikely to be accessed, the sub-block may not be stored in the cache. Associated with each block is a mask which indicates whether sub-blocks of the block are likely to be accessed. When a block is first loaded into the cache, the corresponding mask is cleared and an indication is set for the block to indicate a training mode for the block. Access patterns of the block are then monitored and stored in the mask. If a given sub-block is accessed a predetermined number of times, a bit in the mask is set to indicate that the sub-block is likely to be accessed. When a block is evicted from the cache, the mask is also transferred for storage and only the sub-blocks identified by the mask as being likely to be accessed may be transferred for storage. If previously evicted data is restored to the cache, a previously stored mask is accessed to determine which of the sub-blocks are predicted likely to be accessed. The lower level storage may then transfer only those sub-blocks predicted likely to be accessed to the cache.
1. A hierarchical memory system comprising:
a cache coupled to one or more processing cores, wherein the cache is configured to convey a request for a first sub-block within a first cache line comprising a plurality of sub-blocks of data;
a lower-level memory configured to store a cache line which is evicted from the cache;
wherein, in response to receiving said request, the lower-level memory is further configured to:
access a mask corresponding to the first cache line;
identify one or more sub-blocks within the first cache line which are predicted by the mask to be accessed; and
transfer to the cache only the first sub-block and said one or more sub-blocks;
wherein the cache, in response to detecting an access to a second cache line within the cache, is further configured to:
identify a particular sub-block of the second cache line which is addressed by the access; and
store an indication in an entry in a first mask which corresponds to the particular sub-block, wherein the first mask comprises a separate entry for each of the sub-blocks within the second cache line, said indication comprising a prediction that the sub-block is likely to be accessed while the second cache line is present in the cache;
wherein in response to determining a selected cache line stored within the cache is to be evicted, the cache is further configured to access a selected word mask corresponding to the selected cache line prior to evicting the selected cache line, and then transfer to the lower-level memory only those sub-blocks of the selected cache line which are identified by the selected word mask as being predicted likely to be accessed.
2. The memory system of claim 1 , wherein the cache is configured to store said indication in the first mask in response to detecting a training mode corresponding to the second cache line.
3. The memory system as recited in claim 2 , wherein the lower-level memory is further configured to transfer to the cache said one or more sub-blocks, in response to determining said mask comprises a valid prediction.
4. The memory system of claim 2 , wherein in response to detecting an access to a third cache line in the cache, the cache is further configured to:
determine whether a training mode is indicated for the third cache line; and
in response to determining a training mode is not indicated for the third cache line:
access a third mask corresponding to the third cache line in order to determine whether a valid sub-block addressed by the access to the third cache line is present in the cache; and
indicate a cache miss, in response to determining a valid sub-block addressed by the access to the third cache line is not present in the cache.
5. The memory system of claim 1 , wherein each sub-block comprises a word size of a processor, and wherein each cache line comprises multiple words.
6. A method in a hierarchical memory system comprising:
conveying a request from a cache for a first sub-block within a first cache line comprising a plurality of sub-blocks of data;
accessing a mask corresponding to the first cache line, in response to receiving said request;
identifying one or more sub-blocks of the cache line which are predicted by the mask to be accessed; and
transferring to the cache only the first sub-block and said one or more sub-blocks;
wherein in response to detecting an access to a second cache line within the cache, the method further comprises:
identifying a particular sub-block of the second cache line which is addressed by the access; and
storing an indication in an entry in a first mask which corresponds to the particular sub-block, wherein the mask comprises a separate entry for each of the sub-blocks within the second cache line, said indication comprising a prediction that the sub-block is likely to be accessed while the second cache line is present in the cache;
wherein in response to determining a selected cache line stored within the cache is to be evicted, the method further comprises:
accessing a selected word mask corresponding to the selected cache line prior to evicting the selected cache line; and then
transferring to a lower-level memory only those sub-blocks of the selected cache line which are identified by the selected word mask as being predicted likely to be accessed.
7. The method of claim 6 , wherein storing said indication in the first mask is in further response to detecting a training mode corresponding to the second cache line.
8. The method as recited in claim 7 , wherein transferring to the cache said one or more sub-blocks is in further response to determining said mask comprises a valid prediction.
9. The method of claim 7 , wherein in response to detecting an access to a third cache line in the cache, the method further comprises the cache determining whether a training mode is indicated for the third cache line, and in response to determining a training mode is not indicated for the third cache line:
accessing a third mask corresponding to the third cache line in order to determine whether a valid sub-block addressed by the access to the third cache line is present in the cache; and
indicating a cache miss, in response to determining a valid sub-block addressed by the access to the third cache line is not present in the cache.
10. The method of claim 6 , wherein each sub-block comprises a word size of a processor, and wherein each cache line comprises multiple words.
11. A computing system comprising:
a main memory;
a processing unit comprising a cache located between one or more processing cores and the main memory, wherein the cache is configured to convey a request for a first sub-block within a first cache line comprising a plurality of sub-blocks of data;
a lower-level memory, coupled to the processing unit, configured to store a cache line which is evicted from the first cache;
wherein, in response to receiving said request, the lower-level memory is further configured to:
access a mask corresponding to the first cache line;
identify one or more sub-blocks within the first cache line which are predicted by the mask to be accessed; and
transfer to the cache only the first sub-block and said one or more sub-blocks;
wherein the cache, in response to detecting an access to a second cache line within the cache, is further configured to:
identify a particular sub-block of the second cache line which is addressed by the access; and
store an indication in an entry in a first mask which corresponds to the particular sub-block, wherein the first mask comprises a separate entry for each of the sub-blocks within the second cache line, said indication comprising a prediction that the sub-block is likely to be accessed while the second cache line is present in the cache;
wherein in response to determining a selected cache line stored within the cache is to be evicted, the cache is further configured to access a selected word mask corresponding to the selected cache line prior to evicting the selected cache line, and then transfer to the lower-level memory only those sub-blocks of the selected cache line which are identified by the selected word mask as being predicted likely to be accessed.
12. The computing system of claim 11 , wherein the cache is configured to store said indication in the first mask in response to detecting a training mode corresponding to the second cache line.
13. The computing system as recited in claim 12 , wherein the lower-level memory is further configured to transfer to the cache said one or more blocks, in response to determining said mask comprises a valid prediction.
14. The computing system of claim 12 , wherein in response to detecting an access to a third cache line in the cache, the cache is further configured to:
determine whether a training mode is indicated for the third cache line; and
in response to determining a training mode is not indicated for the third cache line:
access a third mask corresponding to the third cache line in order to determine whether a valid sub-block addressed by the access to the third cache line is present in the cache; and
indicate a cache miss, in response to determining a valid sub-block addressed by the access to the third cache line is not present in the cache.
15. The memory system of claim 1 , wherein said access comprises a read access.