IP Library › Granted Patent US 12,493,481
Granted Patent B2
US 12,493,481 · App. 17/529,891 · Granted Dec 9, 2025

Live migration between hosts of a virtual machine connection to a host interface

Inventors: Jonas Alexander Pfefferle (Rueschlikon, CH); Nikolas Ioannou (Zurich, CH); Jose Gabriel Castanos (Yorktown Heights, NY); Danny Harnik (Tel Aviv, IL); Gautam H. Shah (Shrewsbury, MA)
Assignee: International Business Machines Corporation
G06F9/45558G06F9/4856G06F2009/4557G06F2009/45579G06F2009/45595
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,493,481
App. No.
17/529,891
Granted
Dec 9, 2025
Kind
B2
Abstract

Live migration of a virtual machine (VM) includes establishing multipath connections between the VM and functions of host interface on a source host. The multipath connections include a passthrough path and a software-virtualized (or emulated) path provided by a hypervisor of the source host. A failover of the passthrough path to the emulated path is executed, and a state of the emulated path is thereafter saved. On a host interface of a destination host, functions corresponding to those of the source host are exposed. The VM is then migrated from the source host to the destination host. The VM resumes host interface communication with the host interface of the destination host from the saved state via an emulated path provided by a hypervisor of the destination host. After resuming communication, a passthrough path of communication between the VM and the host interface of the destination host is established.

Claims (55)

1 . A method of live migration of a virtual machine in a data processing environment including a source host and a destination host, the method comprising:

establishing multipath connections between the virtual machine on the source host and a first function and a second function of a first host interface of the source host, the multipath connections including a passthrough path and an emulated path provided by a hypervisor of the source host;

executing, on the source host, a failover of the passthrough path to the emulated path, wherein the executing of the failover includes:

determining that all commands from the virtual machine to the first function of the first host interface, communicated via the passthrough path are completed; and

executing a hot un-plug of the passthrough path on the source host based on the determining that all the commands from the virtual machine to the first function communicated via the passthrough path are completed;

saving, based on the executing of the failover, a state of the emulated path;

exposing, on a second host interface of the destination host, a first function and a second function corresponding to the first function and the second function of the first host interface of the source host;

migrating, based on the exposing, the virtual machine from the source host to the destination host;

resuming, on the destination host, communication of the virtual machine to the second function of the second host interface from the saved state of the emulated path of the source host, wherein

the communication is resumed via an emulated path provided by a hypervisor of the destination host; and

establishing, after the resuming of the communication, a passthrough path of communication between the virtual machine and the first function of the second host interface of the destination host.

2 . The method of claim 1 , wherein the establishing of the multipath connections between the virtual machine on the source host and the first function and the second function of the first host interface of the source host includes:

automatically establishing, by an operating system of the virtual machine, the multipath connections based on sharing of a shared storage volume by multiple controllers of the virtual machine.

3 . The method of claim 1 , further comprising:

un-plugging, after the establishing of the passthrough path of the communication between the virtual machine and the first function of the second host interface of the destination host, the emulated path on the destination host.

4 . The method of claim 1 , wherein the emulated path on the source host includes a back-end driver and a paravirtualized device.

5 . The method of claim 1 , wherein the first host interface comprises a Non-Volatile Memory Express (NMVe) controller.

6 . The method of claim 1 , wherein the passthrough path on the source host comprises a Peripheral Component Interconnect Express (PCIe) passthrough path.

7 . A data processing system serving as a source host, the data processing system comprising:

a processor;

a first host interface coupled to the processor; and

a storage system coupled to the processor, wherein the storage system includes program code executable by the processor that causes the data processing system to:

establish multipath connections between a virtual machine on the source host and a first function and a second function of the first host interface of the source host, the multipath connections including a passthrough path and an emulated path provided by a hypervisor of the source host;

execute a failover of the passthrough path to the emulated path, wherein the execution of the failover includes:

determination that all commands from the virtual machine to the first function of the first host interface, communicated via the passthrough path are completed; and

execution of a hot un-plug of the passthrough path on the source host based on the determination that all the commands from the virtual machine to the first function communicated via the passthrough path are completed;

save, based on the execution of the failover, a state of the emulated path;

co-ordinate with a destination host to:

expose, on a second host interface of the destination host, a first function and a second function corresponding to the first function and the second function of the first host interface of the source host;

migrate, based on the exposure of the first function and the second function on the second host interface, the virtual machine from the source host to the destination host; and

resume, on the destination host, communication of the virtual machine to the second host interface from the saved state of the emulated path of the source host,

wherein the communication is resumed via an emulated path provided by a hypervisor of the destination host.

8 . The data processing system of claim 7 , wherein the establishment of the multipath connections between the virtual machine on the source host and the first function and the second function of the first host interface of the source host includes:

automatic establishment, by an operating system of the virtual machine, of the multipath connections based on multiple controllers of the virtual machine that share a shared storage volume.

9 . The data processing system of claim 7 , wherein the emulated path on the source host includes a back-end driver and a paravirtualized device.

10 . The data processing system of claim 7 , wherein the first host interface comprises a Non-Volatile Memory Express (NMVe) controller.

11 . The data processing system of claim 7 , wherein the passthrough path on the source host comprises a Peripheral Component Interconnect Express (PCIe) passthrough path.

12 . The data processing system of claim 7 , wherein the program code further causes the data processing system: update, on the source host, a state of the passthrough path to inaccessible, and wherein the state of the passthrough path on the source host is updated to inaccessible prior to the execution of the failover on the source host.

13 . A program product, comprising:

a storage device; and

program code, stored within the storage device, which when executed by a processor of a data processing system serving a source host causes the data processing system to perform:

establishing multipath connections between a virtual machine on the source host and a first function and a second function of a first host interface of the source host, the multipath connections including a passthrough path and an emulated path provided by a hypervisor of the source host;

executing a failover of the passthrough path to the emulated path, wherein the executing of the failover includes:

determining that all commands from the virtual machine to the first function of the first host interface, communicated via the passthrough path are completed; and

executing a hot un-plug of the passthrough path on the source host based on the determining that all the commands from the virtual machine to the first function communicated via the passthrough path are completed;

saving, based on the executing of the failover, a state of the emulated path;

coordinating with a destination host to;

expose, on a second host interface of the destination host, a first function and a second function corresponding to the first function and second function of the first host interface of the source host;

migrate, based on the expose of the first function and the second function on the second host interface, the virtual machine from the source host to the destination host; and

resume, on the destination host, communication, of the virtual machine to the second host interface from the saved state of the emulated path of the source host, wherein the communication is resumed via an emulated path provided by a hypervisor of the destination host.

14 . The program product of claim 13 , wherein the establishing of the multipath connections between the virtual machine on the source host and the first function and the second function of the first host interface of the source host includes:

automatically establishing, by an operating system of the virtual machine, the multipath connections based on sharing, of a shared storage volume by multiple controllers of the virtual machine.

15 . The program product of claim 13 , wherein the emulated path on the source host includes a back-end driver and a paravirtualized device.

16 . The program product of claim 13 , wherein the first host interface comprises a Non-Volatile Memory Express (NMVe) controller.

17 . The program product of claim 13 , wherein the passthrough path on the source host comprises a Peripheral Component Interconnect Express (PCIe) passthrough path.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 18, 2021
From: PFEFFERLE, JONAS ALEXANDER; IOANNOU, NIKOLAS; CASTANOS, JOSE GABRIEL; HARNIK, DANNY; SHAH, GAUTAM H.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 058153/0695 →
Continuity (1)
Related Publication 20230153140A1 · May 18, 2023
References Cited (34)
US 11169835B1 · Duong · 2021 [cited by examiner]
US 20080222375A1 · Kotsovinos · 2008 [cited by examiner]
US 20120017031A1 · Mashtizadeh · 2012 [cited by examiner]
US 20120042034A1 · Goggin · 2012 [cited by examiner]
US 20130305246A1 · Goggin · 2013 [cited by examiner]
US 20140229697A1 · Colbert · 2014 [cited by examiner]
US 20150220354A1 · Nair · 2015 [cited by examiner]
US 20150242227A1 · Nair · 2015 [cited by examiner]
US 20150317177A1 · Hussain · 2015 [cited by examiner]
US 20180139101A1 · Puri · 2018 [cited by examiner]
US 20180157561A1 · Venkatesh · 2018 [cited by examiner]
US 20180349161A1 · Jain · 2018 [cited by examiner]
US 20190065096A1 · Sterin · 2019 [cited by examiner]
US 20190163521A1 · Lee · 2019 [cited by examiner]
US 20190377594A1 · Xie · 2019 [cited by examiner]
US 20200150997A1 · Chang · 2020 [cited by examiner]
US 20200167181A1 · Tsirkin · 2020 [cited by examiner]
US 20200167247A1 · Tsirkin · 2020 [cited by examiner]
US 20210165675A1 · Wang · 2021 [cited by examiner]
US 20210232468A1 · Tidke · 2021 [cited by examiner]
US 20220334989A1 · Bar-Ilan · 2022 [cited by examiner]
US 20230153140A1 · Pfefferle · 2023 [cited by examiner]
WO 2023089387A1 · 2023 [cited by applicant]
Michael Nelson et al., Fast Transparent Migration for Virtual Machines, ATEC '05, Apr. 10, 2005. [cited by applicant]
Live migration, Last updated Jul. 8, 2021 https://cloud.google.com/compute/docs/instances/live-migration. [cited by applicant]
Configuring Device Mapper Multipath, Red Hat Enterprise Linux, Copyright 2021. [cited by applicant]
PCT International Search Authority, International Search Report and Written Opinion, Nov. 28, 2022, 7 pages. [cited by applicant]
“QEMU—the FAST! processor emulator”, retrieved from web https://web.archive.org/web/20210811221752/https://www.qemu.org/, Aug. 11, 2021, 2 pages. [cited by applicant]
Hildenbrand et al. “Virtio-(balloon|pmem|mem): Managing Guest Memory”, KVM Forum, 2020, 21 pages. [cited by applicant]
No Author. “Base Specification”, NVM express, May 13, 2021, 452 pages. [cited by applicant]
Tian et al. “colOMMU: A Virtual IOMMU with Cooperative DMA Buffer Tracking for Efficient Memory Management in Direct I/”, Usenix—The advanced computing systems association, Jul. 15-17, 2020, 15 pages. [cited by applicant]
Wunner Lukas. “The modernization of PCle hotplug in Linux”, retrieved from web https://lwn.net/Articles/767885/, Oct. 8, 2018, 6 pages. [cited by applicant]
Xu et al. “SRVM: Hypervisor Support for Live Migration with Passthrough SR-IOV Network Devices” Acm Sigplan Notices, pp. 65-77, vol. 51, Issue 7. [cited by applicant]
Zheng et al. “Live Migration Support for GPU with SR-IOV”, retrieved from web https://events 19.linuxfoundation.org/wp-content/uploads/2017/12/Live-Migration-Support-for-GPU-with-SRIOV-Challenges-and-Solution-Zheng-Xiao… [cited by applicant]