IP Library › Granted Patent US 12,554,652
Granted Patent B2
US 12,554,652 · App. 17/932,326 · Granted Feb 17, 2026

Virtual machine failover with disaggregated shared memory

Inventors: Daniel Waddington (Morgan Hill, CA); Moshik Hershcovitch (Tel-Aviv, IL)
Assignee: International Business Machines Corporation
G06F12/121G06F9/45558G06F2009/45583G06F2212/1044
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,554,652
App. No.
17/932,326
Granted
Feb 17, 2026
Kind
B2
Abstract

According to an aspect, a computer-implemented method includes operating a program on a virtual machine on a first device having a local cache memory. Based on a determination that an epoch timer has not expired, aspects include writing one or more updates to the local cache memory and transmitting evicted items from the local cache memory to a shared memory device that is separate from the first device. Based on a determination that an epoch timer has expired, aspects include flushing the local cache memory to the shared memory device, transmitting a virtual CPU state of the virtual machine to the shared memory device, and resetting the epoch timer.

Claims (52)

1 . A computer-implemented method comprising:

operating a program on a virtual machine on a first device having a local cache memory, wherein the first device is coupled via a cache-coherent interconnect to a disaggregated shared memory device that is physically separate from the first device, the disaggregated shared memory device storing the entirety of a main memory for the program, and serving as a primary memory for the virtual machine;

based on a determination that an epoch timer has not expired:

writing one or more updates to the local cache memory; and

transmitting, over the cache-coherent interconnect, evicted dirty cache lines from the local cache memory directly to the disaggregated shared memory device;

based on a determination that an epoch timer has expired:

flushing all dirty cache lines from the local cache memory, over the cache-coherent interconnect, to the disaggregated shared memory device;

transmitting a virtual CPU state of the virtual machine to the disaggregated shared memory device; and

resetting the epoch timer;

wherein a duration of the epoch timer is reconfigurable by a hypervisor associated with the virtual machine, and wherein the epoch timer determines the frequency of capturing snapshots of the virtual machine.

2 . The computer-implemented method of claim 1 , wherein a duration of the epoch timer is set by a hypervisor associated with the virtual machine.

3 . The computer-implemented method of claim 1 , further comprising transmitting a transaction begin message to the disaggregated shared memory device based on the epoch timer being reset.

4 . The computer-implemented method of claim 1 , further comprising transmitting a transaction end message to the disaggregated shared memory device based on a determination that the epoch timer has expired.

5 . The computer-implemented method of claim 1 , further comprising:

detecting a failure of the first device; and

transmitting a notification of the failure to a second device, wherein the notification causes the program to execute on the second device.

6 . The computer-implemented method of claim 5 , wherein the second device is configured to access the disaggregated shared memory device.

7 . The computer-implemented method of claim 1 , wherein the disaggregated shared memory device is configured to store a main memory for the program.

8 . The computer-implemented method of claim 1 , wherein the disaggregated shared memory device is configured to update a log corresponding to the main memory for the program based on receiving the virtual CPU state of the virtual machine.

9 . The computer-implemented method of claim 8 , wherein the update to the log marks the previously received updates to the main memory for the program as committed.

10 . A system comprising:

a memory having computer readable instructions; and one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:

operating a program on a virtual machine on a first device having a local cache memory, wherein the first device is coupled via a cache-coherent interconnect to a disaggregated shared memory device that is physically separate from the first device, the disaggregated shared memory device storing the entirety of the main memory for the program, and serving as the primary memory for the virtual machine;

based on a determination that an epoch timer has not expired:

writing one or more updates to the local cache memory; and

transmitting, evicted dirty cache lines from the local cache memory directly to the disaggregated shared memory device;

based on a determination that an epoch timer has expired:

flushing all dirty cache lines from the local cache memory, over the cache coherent interconnect, to the disaggregated shared memory device;

transmitting a virtual CPU state of the virtual machine to the disaggregated shared memory device; and

resetting the epoch timer;

wherein a duration of the epoch timer is reconfigurable by a hypervisor associated with the virtual machine, and wherein the epoch timer determines the frequency of capturing snapshots of the virtual machine.

11 . The system of claim 10 , wherein a duration of the epoch timer is set by a hypervisor associated with the virtual machine.

12 . The system of claim 10 , wherein the operations further comprise transmitting a transaction begin message to the disaggregated shared memory device based on the epoch timer being reset.

13 . The system of claim 10 , wherein the operations further comprise transmitting a transaction end message to the disaggregated shared memory device based on a determination that the epoch timer has expired.

14 . The system of claim 10 , wherein the operations further comprise:

detecting a failure of the first device; and

transmitting a notification of the failure to a second device, wherein the notification causes the program to execute on the second device.

15 . The system of claim 14 , wherein the second device is configured to access the disaggregated shared memory device.

16 . The system of claim 10 , wherein the disaggregated shared memory device is configured to store a main memory for the program.

17 . The system of claim 10 , wherein the disaggregated shared memory device is configured to update a log corresponding to the main memory for the program based on receiving the virtual CPU state of the virtual machine.

18 . The system of claim 17 , wherein the update to the log marks the previously received updates to the main memory for the program as committed.

19 . A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:

operating a program on a virtual machine on a first device having a local cache memory, wherein the first device is coupled via a cache-coherent interconnect to a disaggregated shared memory device that is physically separate from the first device, the disaggregated shared memory device storing the entirety of the main memory for the program, and serving as the primary memory for the virtual machine;

based on a determination that an epoch timer has not expired:

writing one or more updates to the local cache memory; and

transmitting, over the cache-coherent interconnect, evicted dirty cache lines from the local cache memory directly to the disaggregated shared memory device that is separate from the first device;

based on a determination that an epoch timer has expired:

flushing all dirty cache lines from the local cache memory, over the cache-coherent interconnect, to the disaggregated shared memory device;

transmitting a virtual CPU state of the virtual machine to the disaggregated shared memory device; and

resetting the epoch timer;

wherein a duration of the epoch timer is reconfigurable by a hypervisor associated with the virtual machine, and wherein the epoch timer determines the frequency of capturing snapshots of the virtual machine.

20 . The computer program product of claim 19 , wherein a duration of the epoch timer is set by a hypervisor associated with the virtual machine.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 15, 2022
From: WADDINGTON, DANIEL; HERSHCOVITCH, MOSHIK
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 061103/0870 →
Continuity (1)
Related Publication 20240095186A1 · Mar 21, 2024
References Cited (34)
US 10379967B2 · Wade · 2019 [cited by examiner]
US 10838647B2 · Guim Bernat et al. · 2020 [cited by applicant]
US 10969975B2 · Bernat et al. · 2021 [cited by applicant]
US 11016798B2 · Gopalan · 2021 [cited by applicant]
US 11016908B2 · Johns et al. · 2021 [cited by applicant]
US 11036531B2 · Connor et al. · 2021 [cited by applicant]
US 11200168B2 · Johns et al. · 2021 [cited by applicant]
US 11288194B2 · Johns et al. · 2022 [cited by applicant]
US 11288208B2 · Nair et al. · 2022 [cited by applicant]
US 20140068197A1 · Joshi et al. · 2014 [cited by applicant]
US 20150378770A1 · Guthrie et al. · 2015 [cited by applicant]
US 20160210208A1 · Chen · 2016 [cited by applicant]
US 20200371692A1 · Van Doorn et al. · 2020 [cited by applicant]
US 20220382574A1 · Gopalan · 2022 [cited by examiner]
Li et al., ConSnap: Taking Continuous Snapshots for Running State Protection of Virtual Machines, 2014, 2014 20th IEEE International Conference on Parallel and Distributed Systems (ICPADS), pp. 1-8 (Year: 2014). [cited by examiner]
Anonymous, “Page Modification Logging for Virtual Machine Monitor White Paper,” Intel, 331872-001, Jan. 2015, 7 pages. [cited by applicant]
International Search Report and Written Opinion; Application No. PCT/EP2023/074855; Filed: Sep. 11, 2023; Date of mailing: Dec. 21, 2023; 14 pages. [cited by applicant]
Anonymously, “A Method to Improve Compute Node Disaster Recovery Efficiency in Large-Scale Cloud Environment.”, IP.Com. Mar. 30, 2019. 5 Pages. [cited by applicant]
Anonymously, “Method and System for Automatically Testing High Availability (HA) Failover Procedure in Virtualization Based Environment.” IP.Com, Mar. 10, 2011. 3 Pages. [cited by applicant]
Anonymously, “Persistent Memory Development Kit.” https://pmem.io/pmdk/.Jul. 14, 2022, 7 Pages. [cited by applicant]
Anonymously, System and Method for In-Memory High Availabilty/Disaster Recovery Solution. IP.Com, Mar. 30, 2019. Jul. 4, 2013. 6 Pages. [cited by applicant]
Anonymously, Method to Maintain Consistent Data on Recovery Site in Hybrid Cloud Environment. IP.Com, Jan. 23, 2022. 6 Pages. [cited by applicant]
Basu, A. et al.,“Efficient Virtual Memory for Big Memory Servers.” google.com, 2013. 12 Pages. [cited by applicant]
Calciu, I. et al., “Rethinking Software Runtimes for Disaggregated Memory.” ASPLOS '21, Apr. 19-23, 2021, Virtual, USA. Apr. 2021. pp. 1-14. [cited by applicant]
D. Ustiugov et al., “Benchmarking, Analysis, and Optimization of Serverless Function Snapshots.” ASPLOS '21, Apr. 19-23, 2021, Virtual, USA. 14 Pages. [cited by applicant]
J. Ren et al., “ThyNVM: Enabling Software-Transparent Crash Consistency in Persistent Memory Systems.” IEEE Xplore. Jul. 14, 2022, 14 Pages. [cited by applicant]
J. Zhao et al., “Kiln: Closing the Performance Gap Between Systems With and Without Persistence Support.” IEEE Xplore. Jul. 14, 2022. 12 Pages. [cited by applicant]
K. Doshi et al., “Atomic Persistence for SCM with a Non-intrusive Backend Controller.” IEEE Xplore. Jul. 14, 2022, 13 Pages. [cited by applicant]
K. Tang et al.,“DV-NVLLC: Efficiently guaranteeing crash consistency in persistent memory via dynamic versioning.” IEEE Xplore. Jul. 14, 2022, 8 Pages. [cited by applicant]
Liu, L. et al., “Memory Disaggregation: Research Problems and Opportunities.” Google.com, 1999. 10 Pages. [cited by applicant]
M.Cai et al.,“HOOP: Efficient Hardware-Assisted Out-of-Place Update for Non-Volatile Memory.” Google.com, 13 Pages. [cited by applicant]
V. Kommareddy et al. “Page Migration Support for Disaggregated Non-Volatile Memories.” MEMSYS' Sep. 30-Oct 3, 2019, 6 Pages. [cited by applicant]
VMWare, “vSphere Troubleshooting”, google.com. May 2018. 112 Pages. [cited by applicant]
Zhou, Y. et al., “Fast Cluster Failover Using Virtual Memory-Mapped Communication.” ACM International Conference on Supercomputing, 1999. 11 Pages. [cited by applicant]