Allocating and accessing memory pages with near and far memory blocks from heterogeneous memories
A heterogeneous memory system is implemented using a low-latency near memory (NM) and a high-latency far memory (FM). Pages in the memory system include NM blocks stored in the NM and FM blocks stored in the FM. A page is assigned to a region in the memory system based on the proportion of NM blocks in the page. When accessing a block, the block address is used to determine a region of the memory system, and a block offset is used to determine whether the block is stored in NM or FM. The memory system may observe memory accesses to determine the access statistics of the page and the block. Based on a page's hotness and access density, the page may be migrated to a different region. Based on a block's hotness, the block may be migrated between NM and FM allocated to the page.
1. A computer-implemented method, comprising:
performing, by a heterogeneous memory system including a low-latency near memory (NM) and a high-latency far memory (FM):
allocating a page, wherein the page contains a set of NM blocks stored in the NM and a set of FM blocks stored in the FM, and the page is assigned to one region of the plurality of regions based at least in part on a proportion of NM blocks in the page;
receiving an access request for a block in the page indicating an address for the block;
determining the region of the page based at least in part on the address;
determining whether the block resides in NM or FM based at least in part on a block offset within the address; and
performing the access request for the block.
2. The computer-implemented method of claim 1 , wherein determining the region of the page based at least in part on a comparison of the address and contents of one or more region boundary registers.
3. The computer-implemented method of claim 1 , wherein determining whether the block resides in NM or FM is performed using an offset comparator for the region.
4. The computer-implemented method of claim 1 , further comprising:
performing, by an operating system of the heterogeneous memory system:
determining an access density for the page based at least in part on a number of distinct blocks in the page that are accessed in a time period;
determining a different proportion of blocks in the page to be stored in NM based at least in part on the access density; and
migrating the page to a different region of the plurality of regions based at least in part on the different proportion of blocks in the page to be stored in NM.
5. The computer-implemented method of claim 4 , wherein determining the access density for the page comprises:
tracking, in a translation lookaside buffer (TLB), individual blocks in the page that are accessed while the page is in a cache; and
counting a number of the individual blocks that are accessed when the page is evicted from the cache.
6. The computer-implemented method of claim 4 , wherein migrating the page to a different region is performed as part of a migration of a plurality of pages, wherein the migration is performed for successive epochs.
7. The computer-implemented method of claim 4 , further comprising:
determining a page hotness indicator of the page based at least in part on an access count of the page in the time period; and
determining whether to migrate the page based at least in part on the page hotness indicator.
8. The computer-implemented method of claim 7 , wherein determining the page hotness indicator comprises:
comparing the access count with a dynamic access count threshold that changes with different time periods.
9. The computer-implemented method of claim 4 , further comprising:
migrating, by a hardware memory controller of the heterogeneous memory system, the block from FM allocated for the page to NM allocated for the page, in response to an access of the block.
10. The computer-implemented method of claim 9 , wherein migrating the block from FM to NM comprises updating a remap entry corresponding to the block to indicate a location of the block in the NM.
11. The computer-implemented method of claim 10 , wherein:
migrating the block from FM to NM comprises storing the block in a NM row; and
updating the remap entry comprises updating one or more bits stored in the NM row.
12. A system, comprising:
one or more hardware processors implementing a heterogeneous memory system, comprising:
a low-latency near memory (NM) and a high-latency far memory (FM);
an operating system configured to execute on the one or more hardware processors to:
allocate a page of memory, wherein the page contains a set of NM blocks stored in the NM and a set of FM blocks stored in the FM, and the page is assigned to one region of a plurality of regions based at least in part on a proportion of NM blocks in the page;
receive an access request for a block in the page indicating an address for the block and forward the access request to a memory system controller coupled to the NM and the FM; and
the memory system controller, configured to:
determine the region of the page based at least in part on the address;
determine whether the block resides in NM or FM based at least in part on a block offset within the address; and
perform the access request for the block.
13. The system of claim 12 , wherein:
to determine the region of the page, the memory system controller is configured to compare the address with contents of one or more region boundary registers; and
to determine whether the block resides in NM or FM, the memory system controller is configured to apply an offset comparator for the region to the block offset within the address.
14. The system of claim 12 , wherein the operating system is configured to:
determine an access density for the page based at least in part on a number of distinct blocks in the page that are accessed in a time period;
determine a page hotness indicator for the page based at least in part on an access count of the page in the time period;
migrate the page to a different region of the plurality of regions based at least in part on the access density and the page hotness indicator.
15. The system of claim 12 , wherein the memory system controller is configured to:
in response to an access of the block, migrate the block from FM allocated for the page to NM allocated for the page.
16. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors of a heterogeneous memory system, cause the heterogeneous memory system to implement an operating system configured to:
allocate a page of memory by the operating system, wherein the page includes blocks to be stored in at least two different types of memories, including a proportion of the blocks stored in a low-latency near memory (NM) and another proportion of the blocks stored in a high-latency far memory (FM); and
assign the page to one region of a plurality of regions of memory based at least in part on the proportion of the blocks in the page stored in the low-latency NM, wherein each region is configured to store pages with a distinct ratio of NM blocks to FM blocks.
17. The non-transitory computer-accessible storage medium of claim 16 , wherein the program instructions when executed on the one or more processors cause the heterogeneous memory system to:
determine an access density for the page based at least in part on a number of distinct blocks in the page that are accessed in a time period; and
determine a different proportion of blocks in the page to be stored in NM based at least in part on the access density; and
migrate the page to a different region of the plurality of regions based at least in part on the different proportion of blocks in the page to be stored in NM.
18. The non-transitory computer-accessible storage medium of claim 17 , wherein to determine the access density for the page, the program instructions when executed on the one or more processors cause the heterogeneous memory system to:
obtain from a translation lookaside buffer (TLB) a count of a number of distinct blocks within the page that are accessed while the page is cached.
19. The non-transitory computer-accessible storage medium of claim 17 , wherein to migrate the page to a different region, the program instructions when executed on the one or more processors cause the heterogeneous memory system to perform the migration as part of a periodic migration of a plurality of pages across the plurality of regions.
20. The non-transitory computer-accessible storage medium of claim 17 , wherein the program instructions when executed on the one or more processors cause the heterogeneous memory system to:
determine a page hotness indicator of the page based at least in part on an access count of the page in the time period; and
determine whether to migrate the page based at least in part a comparison of the access count with a dynamic access count threshold that changes with different time periods.