IP Library Granted Patent US 11,928,510
Granted Patent B2
US 11,928,510 · App. 17/324,447 · Granted Mar 12, 2024

Increasing page sharing on non-uniform memory access (NUMA)-enabled host systems

Inventors: Qasim Ali (Santa Clara, CA); Arunachalam Ramanathan (Union City, CA); Rajesh Venkatasubramanian (San Jose, CA)
Assignee: VMware LLC
G06F9/5016G06F9/45558G06F9/5077G06F12/0882G06F12/1009G06F2009/45583G06F2009/45595G06F2212/2542
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,928,510
App. No.
17/324,447
Granted
Mar 12, 2024
Kind
B2
Abstract

In one set of embodiments, a hypervisor of a host system can determine that a delta between local and remote memory access latencies for each of a subset of NUMA nodes of the host system is less than a threshold. In response, the hypervisor can enable page sharing across the subset of NUMA nodes, where enabling page sharing comprises associating the subset of NUMA nodes with a single page sharing table, and where the single page sharing table holds entries identifying host physical memory pages of the host system that are shared by virtual machines (VMs) placed on the subset of NUMA nodes.

Claims (81)

1. A method comprising:

determining, by a hypervisor of a host system, that a delta between local and remote memory access latencies for each of a subset of non-uniform memory access (NUMA) nodes of the host system is less than a threshold; and

in response to the determining, enabling, by the hypervisor, page sharing across the subset of NUMA nodes, wherein the enabling comprises associating the subset of NUMA nodes with a single page sharing table, and wherein the single page sharing table holds entries identifying host physical memory pages of the host system that are shared by virtual machines (VMs) placed on the subset of NUMA nodes.

2. The method of claim 1 further comprising:

maintaining a copy of a parent VM on every NUMA node of the host system; and

at a time of creating an instant clone of the parent VM on a first NUMA node of the host system, creating the instant clone from the copy of the parent VM maintained on the first NUMA node, the creating causing the instant clone to share one or more host physical memory pages with the copy of the parent VM maintained on the first NUMA node.

3. The method of claim 1 further comprising:

maintaining a parent VM on a first NUMA node of the host system;

receiving a request to provision an instant clone of the parent VM on the host system;

determining that the instant clone should be placed on a second NUMA node of the host system that is different from the first NUMA node; and

checking whether another instant clone of the parent VM exists on the second NUMA node.

4. The method of claim 3 further comprising, upon determining that another instant clone of the parent VM does not exist on the second NUMA node:

copying a set of host physical memory pages mapped to guest physical memory of the parent VM from the first NUMA node to the second NUMA node, the copying resulting in a set of copied host physical memory pages on the second NUMA node;

mapping guest physical memory of the instant clone to the set of copied host physical memory pages; and

adding entries for the set of copied host physical memory pages to a page sharing table associated with the second NUMA node.

5. The method of claim 4 further comprising, upon determining that another instant clone of the parent VM exists on the second NUMA node:

determining, via the page sharing table associated with the second NUMA node, that the set of copied host physical memory pages on the second NUMA node correspond to the guest physical memory of the parent VM; and

in response to the determining, mapping the guest physical memory of the instant clone to the set of copied host physical memory pages.

6. The method of claim 4 further comprising, prior to copying the host physical memory pages mapped to the guest physical memory of the parent VM from the first NUMA node to the second NUMA node:

determining that yet another instant clone of the parent VM exists on the first NUMA node; and

in response to the determining, migrating, rather than copying, the host physical memory pages mapped to the guest physical memory of the parent VM from the first NUMA node to the second NUMA node.

7. The method of claim 1 further comprising:

creating an instant clone of a parent VM on a first NUMA node in the subset of NUMA nodes, the parent VM being maintained on a second NUMA node in the subset of NUMA nodes;

identifying a guest physical memory page of the instant clone that is being frequently accessed during runtime of the instant clone; and

copying, from the first NUMA node to the second NUMA node, a host physical memory page on the first NUMA node that is mapped to the guest physical memory page, the copying resulting in a copied host physical memory page on the second NUMA node; and

mapping the guest physical memory page of the instant clone to the copied host physical memory page on the second NUMA node.

8. A non-transitory computer readable storage medium having stored thereon program code executable by a hypervisor of a host system, the program code embodying a method comprising:

determining that a delta between local and remote memory access latencies for each of a subset of non-uniform memory access (NUMA) nodes of the host system is less than a threshold; and

in response to the determining, enabling page sharing across the subset of NUMA nodes, wherein the enabling comprises associating the subset of NUMA nodes with a single page sharing table, and wherein the single page sharing table holds entries identifying host physical memory pages of the host system that are shared by virtual machines (VMs) placed on the subset of NUMA nodes.

9. The non-transitory computer readable storage medium of claim 8 wherein the method further comprises:

maintaining a copy of a parent VM on every NUMA node of the host system; and

at a time of creating an instant clone of the parent VM on a first NUMA node of the host system, creating the instant clone from the copy of the parent VM maintained on the first NUMA node, the creating causing the instant clone to share one or more host physical memory pages with the copy of the parent VM maintained on the first NUMA node.

10. The non-transitory computer readable storage medium of claim 8 wherein the method further comprises:

maintaining a parent VM on a first NUMA node of the host system;

receiving a request to provision an instant clone of the parent VM on the host system;

determining that the instant clone should be placed on a second NUMA node of the host system that is different from the first NUMA node; and

checking whether another instant clone of the parent VM exists on the second NUMA node.

11. The non-transitory computer readable storage medium of claim 10 wherein the method further comprises, upon determining that another instant clone of the parent VM does not exist on the second NUMA node:

copying a set of host physical memory pages mapped to guest physical memory of the parent VM from the first NUMA node to the second NUMA node, the copying resulting in a set of copied host physical memory pages on the second NUMA node;

mapping guest physical memory of the instant clone to the set of copied host physical memory pages; and

adding entries for the set of copied host physical memory pages to a page sharing table associated with the second NUMA node.

12. The non-transitory computer readable storage medium of claim 11 wherein the method further comprises, upon determining that another instant clone of the parent VM exists on the second NUMA node:

determining, via the page sharing table associated with the second NUMA node, that the set of copied host physical memory pages on the second NUMA node correspond to the guest physical memory of the parent VM; and

in response to the determining, mapping the guest physical memory of the instant clone to the set of copied host physical memory pages.

13. The non-transitory computer readable storage medium of claim 11 wherein the method further comprises, prior to copying the host physical memory pages mapped to the guest physical memory of the parent VM from the first NUMA node to the second NUMA node:

determining that yet another instant clone of the parent VM exists on the first NUMA node; and

in response to the determining, migrating, rather than copying, the host physical memory pages mapped to the guest physical memory of the parent VM from the first NUMA node to the second NUMA node.

14. The non-transitory computer readable storage medium of claim 8 wherein the method further comprises:

creating an instant clone of a parent VM on a first NUMA node in the subset of NUMA nodes, the parent VM being maintained on a second NUMA node in the subset of NUMA nodes;

identifying a guest physical memory page of the instant clone that is being frequently accessed during runtime of the instant clone; and

copying, from the first NUMA node to the second NUMA node, a host physical memory page on the first NUMA node that is mapped to the guest physical memory page, the copying resulting in a copied host physical memory page on the second NUMA node; and

mapping the guest physical memory page of the instant clone to the copied host physical memory page on the second NUMA node.

15. A host system comprising:

a hypervisor;

a plurality of non-uniform memory access (NUMA); and

a non-transitory computer readable medium having stored thereon program code that causes the hypervisor to:

determine that a delta between local and remote memory access latencies for each of a subset of the plurality of NUMA nodes is less than a threshold; and

in response to the determining, enable page sharing across the subset of NUMA nodes, wherein the enabling comprises associating the subset of NUMA nodes with a single page sharing table, and wherein the single page sharing table holds entries identifying host physical memory pages of the host system that are shared by virtual machines (VMs) placed on the subset of NUMA nodes.

16. The host system of claim 15 wherein the program code further causes the hypervisor to:

maintain a copy of a parent VM on every NUMA node in the plurality of NUMA nodes; and

at a time of creating an instant clone of the parent VM on a first NUMA node in the plurality of NUMA nodes, create the instant clone from the copy of the parent VM maintained on the first NUMA node, the creating causing the instant clone to share one or more host physical memory pages with the copy of the parent VM maintained on the first NUMA node.

17. The host system of claim 15 wherein the program code further causes the hypervisor to:

maintain a parent VM on a first NUMA node in the plurality of NUMA nodes;

receive a request to provision an instant clone of the parent VM on the host system;

determine that the instant clone should be placed on a second NUMA node in the plurality of NUMA nodes that is different from the first NUMA node; and

check whether another instant clone of the parent VM exists on the second NUMA node.

18. The host system of claim 17 wherein the program code further causes the hypervisor to, upon determining that another instant clone of the parent VM does not exist on the second NUMA node:

copy a set of host physical memory pages mapped to guest physical memory of the parent VM from the first NUMA node to the second NUMA node, the copying resulting in a set of copied host physical memory pages on the second NUMA node;

map guest physical memory of the instant clone to the set of copied host physical memory pages; and

add entries for the set of copied host physical memory pages to a page sharing table associated with the second NUMA node.

19. The host system of claim 18 wherein the program code further causes the hypervisor to, upon determining that another instant clone of the parent VM exists on the second NUMA node:

determine, via the page sharing table associated with the second NUMA node, that the set of copied host physical memory pages on the second NUMA node correspond to the guest physical memory of the parent VM; and

in response to the determining, map the guest physical memory of the instant clone to the set of copied host physical memory pages.

20. The host system of claim 18 wherein the program code further causes the hypervisor to, prior to copying the host physical memory pages mapped to the guest physical memory of the parent VM from the first NUMA node to the second NUMA node:

determine that yet another instant clone of the parent VM exists on the first NUMA node; and

in response to the determining, migrate, rather than copy, the host physical memory pages mapped to the guest physical memory of the parent VM from the first NUMA node to the second NUMA node.

21. The host system of claim 15 wherein the program code further causes the hypervisor to:

create an instant clone of a parent VM on a first NUMA node in the subset of NUMA nodes, the parent VM being maintained on a second NUMA node in the subset of NUMA nodes;

identify a guest physical memory page of the instant clone that is being frequently accessed during runtime of the instant clone; and

copy, from the first NUMA node to the second NUMA node, a host physical memory page on the first NUMA node that is mapped to the guest physical memory page, the copying resulting in a copied host physical memory page on the second NUMA node; and

map the guest physical memory page of the instant clone to the copied host physical memory page on the second NUMA node.

Assignments (2)
CHANGE OF NAME Recorded Feb 27, 2024
From: VMWARE, INC.
To: VMWARE LLC
Reel/Frame 066692/0103 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 19, 2021
From: ALI, QASIM; RAMANATHAN, ARUNACHALAM; VENKATASUBRAMANIAN, RAJESH
To: VMWARE, INC.
Reel/Frame 056287/0700 →