Tag and data configuration for fine-grained cache memory

1. A method for operating a memory having a plurality of banks accessible in parallel, each bank including a plurality of grains accessible in parallel, the method comprising:

identifying a set that stores data for a memory access request, wherein the set is spread across multiple grains of a plurality of grains, and tags of the set are spread across the multiple grains; and

performing operations to satisfy the memory access request, using entries of the set stored across the multiple grains of the plurality of grains, the performing including accessing the tags stored in different grains in parallel to identify whether a hit occurs.

2. The method of claim 1 , wherein a second set is configured in a configuration in which all tags of the second set are stored in a single grain.

3. The method of claim 2 , further comprising performing operations to satisfy a second memory access request that targets the second set, the operations comprising:

accessing first tags of the second set in parallel with first data of the second set, wherein the first tags are stored in a different grain than the first data.

4. The method of claim 3 , further comprising predicting that the first tags and the first data store correct information for the memory access request.

5. The method of claim 4 , further comprising:

in response to either or both of the first tags or the first data not storing correct information for the memory access request, accessing second tags of the second set and second data of the second set.

6. The method of claim 1 , further comprising:

in response to a hit occurring, and not in parallel with accessing the tags, accessing data of the set to satisfy the memory access request.

7. The method of claim 1 , further comprising:

in response to a miss occurring, fetching data from a backing memory and storing the data into the set.

8. A system comprising:

a memory having a plurality of banks accessible in parallel, each bank including a plurality of grains accessible in parallel; and

a memory controller configured to:

identify a set that stores data for a memory access request, wherein the set is spread across multiple grains of a plurality of grains, and tags of the set are spread across the multiple grains; and

perform operations to satisfy the memory access request, using entries of the set stored across the multiple grains of the plurality of grains, the performing including accessing the tags stored in different grains in parallel to identify whether a hit occurs.

9. The system of claim 8 , wherein a second set is configured in a configuration in which all tags of the second set are stored in a single grain.

10. The system of claim 9 , wherein the memory controller is further configured to perform operations to satisfy a second memory access request that targets the second set, the operations comprising:

accessing first tags of the second set in parallel with first data of the second set, wherein the first tags are stored in a different grain than the first data.

11. The system of claim 10 , wherein the memory controller is further configured to predict that the first tags and the first data store correct information for the memory access request.

12. The system of claim 11 , wherein the memory controller is further configured to:

in response to either or both of the first tags or the first data not storing correct information for the memory access request, access second tags of the second set and second data of the second set.

13. The system of claim 8 , wherein the memory controller is further configured to:

in response to a hit occurring, and not in parallel with accessing the tags, accessing data of the set to satisfy the memory access request.

14. The system of claim 8 , wherein the memory controller is further configured to:

in response to a miss occurring, fetching data from a backing memory and storing the data into the set.

15. A computing device comprising:

a cache;

a memory backing the cache; and

a processor configured to store data in the memory and the cache,

wherein the cache includes:

a cache memory having a plurality of banks accessible in parallel, each bank including a plurality of grains accessible in parallel; and

a memory controller configured to:

identify a set that stores data for a memory access request, wherein the set is spread across multiple grains of a plurality of grains, and tags of the set are spread across the multiple grains; and

perform operations to satisfy the memory access request, using entries of the set stored across the multiple grains of the plurality of grains, the performing including accessing the tags stored in different grains in parallel to identify whether a hit occurs.

16. The computing device of claim 15 , wherein a second set is configured in a configuration in which all tags of the second set are stored in a single grain.

17. The computing device of claim 16 , wherein the memory controller is further configured to perform operations to satisfy a second memory access request that targets the second set, the operations comprising:

accessing first tags of the second set in parallel with first data of the second set, wherein the first tags are stored in a different grain than the first data.

18. The computing device of claim 17 , wherein the memory controller is further configured to predict that the first tags and the first data store correct information for the memory access request.

19. The computing device of claim 18 , wherein the memory controller is further configured to:

in response to either or both of the first tags or the first data not storing correct information for the memory access request, access second tags of the second set and second data of the second set.

20. The computing device of claim 15 , wherein the memory controller is further configured to:

in response to a hit occurring, and not in parallel with accessing the tags, accessing data of the set to satisfy the memory access request.