IP Library Granted Patent US 12,298,897
Granted Patent B2
US 12,298,897 · App. 17/665,390 · Granted May 13, 2025

Architectural support for persistent applications

Inventors: Mazen Alwadi (Irbid, JO); Vamsee Reddy Kommareddy (Leander, TX); Clayton Hughes (Albuquerque, NM); Simon David Hammond (Albuquerque, NM); Amro Awad (Morrisville, NC)
Assignees: National Technology & Engineering Solutions of Sandia, LLC; University of Central Florida Research Foundation, Inc.
G06F12/0238G06F8/65G06F2212/304
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,298,897
App. No.
17/665,390
Granted
May 13, 2025
Kind
B2
Abstract

Illustrative embodiments are directed to methods, apparatus and computer program products for caching at least a fraction of data stored in a non-volatile memory in a mirror region of a dynamic random access memory. A memory controller hub of a processor chip coupled to both the non-volatile memory and the dynamic random access memory is configured to, when an update to the dynamic random access memory is cached in the mirror region of the dynamic random access memory, use the memory controller hub to write the update directly to the mirror region of the dynamic random access memory and concurrently mirror the update to the non-volatile memory to provide coherent persistent durability of the update. When a read from the dynamic random access memory is cached in the mirror region of the dynamic random access memory, embodiments can use the memory controller hub to serve the read directly from the mirror region of the dynamic random access memory to optimize read operations of persistent objects.

Claims (53)

1. A method of architectural support of hybrid memory system persistent applications, the method comprising:

caching at least a fraction of data stored in a non-volatile memory in a mirror region of a dynamic random access memory, the dynamic random access memory comprising a mirror mapping table comprising a hardware managed table,

wherein a memory controller hub of a processor chip coupled to both the non-volatile memory and the dynamic random access memory is configured to, when a first update to the dynamic random access memory is cached in the mirror region of the dynamic random access memory, using the memory controller hub to both write the update directly to the mirror region of the dynamic random access memory and concurrently mirror the update to the non-volatile memory providing coherent persistent durability of the update,

wherein the hardware managed table includes for each mirrored page a group ID, a TAG (non-volatile memory page address) and a dynamic random access memory address.

2. The method of claim 1 , when a read from the dynamic random access memory is cached in the mirror region of the dynamic random access memory, using the memory controller hub to serve the read directly from the mirror region of the dynamic random access memory.

3. The method of claim 1 , when a second update to the dynamic random access memory is not cached in the mirror region of the dynamic random access memory, not concurrently mirroring the update to the non-volatile memory.

4. The method of claim 1 , wherein the memory controller hub supports both memory mode and application direct mode.

5. The method of claim 1 , wherein the memory controller hub transparently ensures durability of updates to persistent pages that are also cached in DRAM.

6. The method of claim 1 , wherein the memory controller hub is configured to, when a read from a memory block is cached in the mirror region of the dynamic random access memory and pressure at the dynamic random access memory is above a threshold, bypassing the mirror region of the dynamic random access memory and accessing the read in the non-volatile memory.

7. The method of claim 1 , wherein the memory controller hub is configured to, when a plurality of pages is cached in the mirror region of the dynamic random access memory, access the plurality of pages in the dynamic random access memory using bank level parallelism, instead of accesses to the non-volatile memory.

8. The method of claim 1 , wherein the memory controller hub is configured to implement multi-queue page eviction based on a least recently used algorithm wherein, when a threshold to be mirrored is reached, M least recently used queues of block descriptors are defined wherein each descriptor comprises a block number, a reference counter and a logical expiration time, wherein a page is cached in the mirror region of the dynamic random access memory using the memory controller hub to both write the page directly to the mirror region of the dynamic random access memory and concurrently mirror the page to the non-volatile memory, to keep hot pages in the mirror region of the dynamic random access memory.

9. A method of architectural support of hybrid memory system persistent applications, the method comprising:

caching at least a fraction of data stored in a non-volatile memory in a mirror region of a dynamic random access memory,

wherein a memory controller hub of a processor chip coupled to both the non-volatile memory and the dynamic random access memory is configured to, when an update to the dynamic random access memory is cached in the mirror region of the dynamic random access memory, using the memory controller hub to both write the update directly to the mirror region of the dynamic random access memory and concurrently mirror the update to the non-volatile memory to provide coherent persistent durability of the update,

wherein a mirror mapping table of the dynamic random access memory comprises a hardware managed table including for each mirrored page a group ID, a TAG (non-volatile memory page address) and a dynamic random access memory address.

10. An apparatus for architectural support of hybrid memory system persistent applications, the apparatus comprising:

a processor chip comprising a core, a cache, and a memory controller hub;

a non-volatile memory coupled to the processor chip; and

a dynamic random access memory coupled to the processor chip, the dynamic random access memory comprising a mirror mapping table comprising a hardware managed table, and a mirror region,

wherein the dynamic random access memory caches at least fraction of data stored in the non-volatile memory in the mirror region of the dynamic random access memory,

wherein the memory controller hub is configured to, when a first update to the dynamic random access memory is cached in the mirror region of the dynamic random access memory, using the memory controller hub to both write the update directly to the mirror region of the dynamic random access memory and concurrently mirror the update to the non-volatile memory providing coherent persistent durability of the update,

wherein the hardware managed table includes for each mirrored page a group ID, a TAG (non-volatile memory page address) and a dynamic random access memory address.

11. The apparatus of claim 10 , when a read from the dynamic random access memory is cached in the mirror region of the dynamic random access memory, using the memory controller hub to serve the read directly from the mirror region of the dynamic random access memory.

12. The apparatus of claim 10 , when a second update to the dynamic random access memory is not cached in the mirror region of the dynamic random access memory, not concurrently mirroring the update to the non-volatile memory.

13. The apparatus of claim 10 , wherein the memory controller hub supports both memory mode and application direct mode.

14. The apparatus of claim 10 , wherein the memory controller hub transparently ensures durability of updates to persistent pages that are also cached in DRAM.

15. The apparatus of claim 10 , wherein the memory controller hub is configured to, when a read from a memory block is cached in the mirror region of the dynamic random access memory and pressure at the dynamic random access memory is above a threshold, bypassing the mirror region of the dynamic random access memory and accessing the read in the non-volatile memory.

16. The apparatus of claim 10 , wherein the memory controller hub is configured to, when a plurality of pages is cached in the mirror region of the dynamic random access memory, access the plurality of pages in the dynamic random access memory using bank level parallelism, instead of accesses to the non-volatile memory.

17. The apparatus of claim 10 , wherein the memory controller hub is configured to implement multi-queue page eviction based on a least recently used algorithm wherein, when a threshold to be mirrored is reached, M least recently used queues of block descriptors are defined wherein each descriptor comprises a block number, a reference counter and a logical expiration time, wherein a page is cached in the mirror region of the dynamic random access memory using the memory controller hub to both write the page directly to the mirror region of the dynamic random access memory and concurrently mirror the page to the non-volatile memory, to keep hot pages in the mirror region of the dynamic random access memory.

18. An apparatus for architectural support of hybrid memory system persistent applications, the apparatus comprising:

a processor chip comprising a core, a cache, and a memory controller hub;

a non-volatile memory coupled to the processor chip; and

a dynamic random access memory coupled to the processor chip, the dynamic random access memory comprising a mirror mapping table and a mirror region,

wherein the dynamic random access memory caches at least fraction of data stored in the non-volatile memory in the mirror region of the dynamic random access memory,

wherein the memory controller hub is configured to, when an update to the dynamic random access memory is cached in the mirror region of the dynamic random access memory, using the memory controller hub to both write the update directly to the mirror region of the dynamic random access memory and concurrently mirror the update to the non-volatile memory to provide coherent persistent durability of the update,

wherein the mirror mapping table of the dynamic random access memory comprises a hardware managed table including for each mirrored page a group ID, a TAG (non-volatile memory page address) and a dynamic random access memory address.

19. A computer program product for architectural support of hybrid memory system persistent applications, the computer program product comprising:

a non-transitory computer readable storage media; and

program code, stored on the computer readable storage media, for caching at least a fraction of data stored in a non-volatile memory in a mirror region of a dynamic random access memory, the dynamic random access memory comprising a mirror mapping table comprising a hardware managed table,

wherein a memory controller hub of a processor chip coupled to both the non-volatile memory and the dynamic random access memory is configured to, when an update to the dynamic access random access memory is cached in the mirror region of the dynamic random access memory, use the memory controller hub to both write the update directly to the mirror region of the dynamic random access memory and concurrently mirror the update to the non-volatile memory providing coherent persistent durability of the update,

wherein the hardware managed table includes for each mirrored page a group ID, a TAG (non-volatile memory page address) and a dynamic random access memory address.

20. The computer program product of claim 19 , when a read from the dynamic random access memory is cached in the mirror region of the dynamic random access memory, use the memory controller hub to serve the read directly from the mirror region of the dynamic random access memory.

21. The computer program product of claim 19 , when a second update to the dynamic random access memory is not cached in the mirror region of the dynamic random access memory, not concurrently mirroring the update to the non-volatile memory.

22. The computer program product of claim 19 , wherein the memory controller hub supports both memory mode and application direct mode.

23. The computer program product of claim 19 , wherein the memory controller hub transparently ensures durability of updates to persistent pages that are also cached in DRAM.

24. The computer program product of claim 19 , wherein the memory controller hub is configured to, when a read from a memory block is cached in the mirror region of the dynamic random access memory and pressure at the dynamic random access memory is above a threshold, bypassing the mirror region of the dynamic random access memory and accessing the read in the non-volatile memory.

25. The computer program product of claim 19 , wherein the memory controller hub is configured to, when a plurality of pages is cached in the mirror region of the dynamic random access memory, access the plurality of pages in the dynamic random access memory using bank level parallelism, instead of accesses to the non-volatile memory.

26. The computer program product of claim 19 , wherein the memory controller hub is configured to implement multi-queue page eviction based on a least recently used algorithm wherein, when a threshold to be mirrored is reached, M least recently used queues of block descriptors are defined wherein each descriptor comprises a block number, a reference counter and a logical expiration time, wherein a page is cached in the mirror region of the dynamic random access memory using the memory controller hub to both write the page directly to the mirror region of the dynamic random access memory and concurrently mirror the page to the non-volatile memory, to keep hot pages in the mirror region of the dynamic random access memory.

27. A computer program product for architectural support of hybrid memory system persistent applications, the computer program product comprising:

a non-transitory computer readable storage media; and

program code, stored on the computer readable storage media, for caching at least a fraction of data stored in a non-volatile memory in a mirror region of a dynamic random access memory,

wherein a memory controller hub of a processor chip coupled to both the non-volatile memory and the dynamic random access memory is configured to, when an update to the dynamic random access memory is cached in the mirror region of the dynamic random access memory, use the memory controller hub to both write the update directly to the mirror region of the dynamic random access memory and concurrently mirror the update to the non-volatile memory to provide coherent persistent durability of the update,

wherein a mirror mapping table of the dynamic random access memory comprises a hardware managed table including for each mirrored page a group ID, a TAG (non-volatile memory page address) and a dynamic random access memory address.

Assignments (3)
CONFIRMATORY LICENSE Recorded Jun 12, 2025
From: NATIONAL TECHNOLOGY & ENGINEERING SOLUTIONS OF SANDIA, LLC
To: NNSA
Reel/Frame 071400/0909 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 17, 2022
From: HUGHES, CLAYTON; HAMMOND, SIMON DAVID
To: NATIONAL TECHNOLOGY & ENGINEERING SOLUTIONS OF SANDIA, LLC
Reel/Frame 059294/0826 →
CONFIRMATORY LICENSE Recorded Feb 15, 2022
From: NATIONAL TECHNOLOGY & ENGINEERING SOLUTIONS OF SANDIA, LLC
To: U.S. DEPARTMENT OF ENERGY
Reel/Frame 059011/0648 →
Continuity (1)
Related Publication 20230251963A1 · Aug 10, 2023
References Cited (36)
US 8924629B1 · Call · 2014 [cited by examiner]
US 9335937B2 · Seo · 2016 [cited by examiner]
US 10180792B1 · Gaertner · 2019 [cited by examiner]
US 20070165443A1 · Richardson · 2007 [cited by examiner]
US 20090046501A1 · Ranjan · 2009 [cited by examiner]
US 20110161560A1 · Hutchison · 2011 [cited by examiner]
US 20130198439A1 · Kurotsuchi · 2013 [cited by examiner]
US 20150347026A1 · Thomas · 2015 [cited by examiner]
US 20160041917A1 · Trojanowski · 2016 [cited by examiner]
US 20170228191A1 · Sun · 2017 [cited by examiner]
US 20180113875A1 · Jain · 2018 [cited by examiner]
US 20180329651A1 · Chang · 2018 [cited by examiner]
US 20180373450A1 · Ji · 2018 [cited by examiner]
US 20190384713A1 · De · 2019 [cited by examiner]
US 20200012601A1 · Jeong · 2020 [cited by examiner]
US 20200020360A1 · Kosuru · 2020 [cited by examiner]
US 20200089609A1 · Colline · 2020 [cited by examiner]
US 20210056016A1 · Bulusu · 2021 [cited by examiner]
US 20210117094A1 · Vankamamidi · 2021 [cited by examiner]
US 20210255920A1 · Bernat · 2021 [cited by examiner]
US 20220019547A1 · Shih · 2022 [cited by examiner]
US 20220365707A1 · Kachare · 2022 [cited by examiner]
US 20220413762A1 · Pletka · 2022 [cited by examiner]
Write Through and Write Back in Cache by Aish 2020 (Year: 2020). [cited by examiner]
Stealth Persist Architectural Support for Persistent Applications by Alwadi Feb. 20, 2021 (Year: 2021). [cited by examiner]
Towards Efficient NVDIMM based Heterogeneous Storage by Chen 2019 (Year: 2019). [cited by examiner]
Mirror Making Lock Free Data Structures Persistent by Friedman 2021 (Year: 2021). [cited by examiner]
Intels Optane Two Confusing Modes by Handy 2019 (Year: 2019). [cited by examiner]
MEDAL Scalable DIMM by Huangfu 2019 (Year: 2019). [cited by examiner]
What is NVDIMM TechTarget by Kranz 2017 (Year: 2017). [cited by examiner]
Transactional Database Performance Boosted with Intel Optane by Ogle 2020 (Year: 2020). [cited by examiner]
Improving Bank Level Parallelism for Irregular Applications by Tang 2016 (Year: 2016). [cited by examiner]
The Multi-Queue Replacement Algorithm by Zhou 2001 (Year: 2001). [cited by examiner]
Page Placement in Hybrid Memory Systems by Ramos (Year: 2011). [cited by examiner]
Stealth-Persist: Architectural Support for Persistent Applications in Hybrid Memory Systems by Alwadi (published less than one year before filing by inventors named in this application) (Year: 2021). [cited by examiner]
Alwadi, M. et al., “Stealth-Persist: Architectural Support for Persistent Applications in Hybrid Memory Systems,” presented at the 27th IEEE International Symposium on High-Performance Computer Architecture (HPCA-27) on… [cited by applicant]