Migrating memory pages between non-uniform memory access (NUMA) nodes based on entries in a page modification log
Memory pages can be migrated between non-uniform memory access (NUMA) nodes based on entries in a page modification log according to some examples described herein. In one example, a physical processor can detect a request from a virtual machine to access a memory page. The physical processor can then update a page modification log to include an entry indicating the request. A hypervisor supporting the virtual machine can be configured to detect the request based on the entry in the page modification log and, in response to detecting the request, migrate the memory page from a second NUMA node to a destination NUMA node.
1 . A non-transitory computer-readable medium comprising program code that is executable by a physical processor of a first non-uniform memory access (NUMA) node for causing the physical processor to:
detect a request from a virtual machine to access a memory page, wherein the virtual machine is supported by a hypervisor of the first NUMA node, and wherein the request is detected by the physical processor rather than the hypervisor; and
in response to detecting the request, update a page modification log to include an entry indicating the request, wherein the hypervisor supporting the virtual machine is configured to:
monitor the page modification log over time to detect a change to the page modification log, wherein the change corresponds to the entry; and
based on detecting the change in the page modification log:
access the entry in the page modification log to indirectly detect the request; and
migrate the memory page from a second NUMA node to a destination NUMA node, the second NUMA node being separate from the first NUMA node.
2 . The non-transitory computer-readable medium of claim 1 , wherein the hypervisor is configured to migrate the memory page from the second NUMA node to the destination NUMA node by:
determining that the memory page exists at a source memory address on the second NUMA node;
copying the memory page from the source memory address to a destination memory address on the destination NUMA node, the destination NUMA node being different than the second NUMA node;
determining a guest physical address associated with the memory page, the guest physical address being an address of the memory page in a virtual memory of the virtual machine; and
generating a page table entry, within a host page table of the hypervisor, that maps the guest physical address in the virtual memory to the destination memory address on the destination NUMA node.
3 . The non-transitory computer-readable medium of claim 2 , wherein the hypervisor is further configured to, prior to copying the memory page from the source memory address to the destination memory address:
select the destination NUMA node for storing the memory page based on a physical distance between the first NUMA node and the destination NUMA node.
4 . The non-transitory computer-readable medium of claim 3 , wherein the destination NUMA node is a third NUMA node, the third NUMA node being different from the first NUMA node and the second NUMA node, and the third NUMA node being physically located closer to the first NUMA node than the second NUMA node in a distributed NUMA node architecture.
5 . The non-transitory computer-readable medium of claim 1 , wherein the destination NUMA node is the first NUMA node.
6 . The non-transitory computer-readable medium of claim 1 , wherein the page modification log is maintained by the physical processor and stored in a physical memory of the first NUMA node.
7 . The non-transitory computer-readable medium of claim 1 , wherein the virtual machine includes a plurality of virtual processors that are executable on a plurality of physical processors of a plurality of NUMA nodes.
8 . The non-transitory computer-readable medium of claim 7 , wherein the entry includes a processor identifier, the processor identifier being a unique identifier of an originating processor of the request, and wherein the hypervisor is further configured to:
determine the processor identifier included in the entry;
access a mapping that correlates processor identifiers to NUMA node identifiers;
identify a correlation in the mapping between the processor identifier and a NUMA node identifier, wherein the NUMA node identifier corresponds to a particular NUMA node of the plurality of NUMA nodes, and wherein the correlation indicates that the particular NUMA node includes the originating processor; and
select the destination NUMA node for the memory page based on the particular NUMA node.
9 . The non-transitory computer-readable medium of claim 7 , wherein a plurality of processor identifiers are configured to uniquely identify the plurality of virtual processors, and wherein the hypervisor is configured to:
generate a mapping that correlates the plurality of processor identifiers to a plurality of guest physical addresses (GPAs) associated with the memory page, wherein each virtual processor of the plurality of virtual processors is configured to request the memory page from a corresponding GPA of the plurality of GPAs; and
subsequent to generating the mapping:
detect the request based on the entry in the page modification log, the entry including a guest physical address associated with the memory page;
determine the guest physical address included in the entry;
identify a correlation in the mapping between the guest physical address and a processor identifier, the correlation indicating that the request was issued by a virtual processor associated with the processor identifier;
identify a particular NUMA node, of the plurality of NUMA nodes, that includes the virtual processor; and
select the destination NUMA node for the memory page based on the particular NUMA node.
10 . The non-transitory computer-readable medium of claim 1 , further comprising program code that is executable by the physical processor for causing the physical processor to update the page modification log to include the entry associated with the request, without triggering an exit of the virtual machine.
11 . A method comprising:
detecting, by a physical processor of a first non-uniform memory access (NUMA) node, a request from a virtual machine to access a memory page, wherein the virtual machine is supported by a hypervisor of the first NUMA node, and wherein the request is detected by the physical processor rather than the hypervisor; and
in response to detecting the request, updating, by the physical processor, a page modification log to include an entry indicating the request, wherein the hypervisor supporting the virtual machine is configured to:
monitor the page modification log over time to detect a change to the page modification log, wherein the change corresponds to the entry; and
based on detecting the change in the page modification log:
access the entry in the page modification log to indirectly detect the request; and
migrate the memory page from a second NUMA node to a destination NUMA node, the second NUMA node being separate from the first NUMA node.
12 . The method of claim 11 , wherein the hypervisor is configured to migrate the memory page from the second NUMA node to the destination NUMA node by:
determining that the memory page exists at a source memory address on the second NUMA node;
copying the memory page from the source memory address to a destination memory address on the destination NUMA node, the destination NUMA node being different than the second NUMA node;
determining a guest physical address associated with the memory page, the guest physical address being an address of the memory page in a virtual memory of the virtual machine; and
generating a page table entry, within a host page table of the hypervisor, that maps the guest physical address in the virtual memory to the destination memory address on the destination NUMA node.
13 . The method of claim 11 , wherein the page modification log is maintained by the physical processor and stored in a physical memory of the first NUMA node.
14 . The method of claim 11 , wherein the virtual machine includes a plurality of virtual processors that are executing on a plurality of physical processors of a plurality of NUMA nodes.
15 . The method of claim 14 , wherein the entry includes a processor identifier, the processor identifier being a unique identifier of an originating processor of the request, and wherein the hypervisor is further configured to:
determine the processor identifier included in the entry;
access a mapping that correlates processor identifiers to NUMA node identifiers:
identify a correlation in the mapping between the processor identifier and a NUMA node identifier, wherein the NUMA node identifier corresponds to a particular NUMA node of the plurality of NUMA nodes, and wherein the correlation indicates that the particular NUMA node includes the originating processor; and
select the destination NUMA node for the memory page based on the particular NUMA node.
16 . The method of claim 14 , wherein a plurality of processor identifiers are configured to uniquely identify the plurality of virtual processors, and wherein the hypervisor is configured to:
generate a mapping that correlates the plurality of processor identifiers to a plurality of guest physical addresses (GPAs) associated with the memory page, wherein each virtual processor of the plurality of virtual processors is configured to request the memory page from a corresponding GPA of the plurality of GPAs; and
subsequent to generating the mapping:
detect the request based on the entry in the page modification log, the entry including a guest physical address associated with the memory page;
determine the guest physical address associated with the request;
identify a correlation in the mapping between the guest physical address and a processor identifier, the correlation indicating that the request was issued by a virtual processor associated with the processor identifier;
identify a particular NUMA node, of the plurality of NUMA nodes, that includes the virtual processor; and
select the destination NUMA node for the memory page based on the particular NUMA node.
17 . The method of claim 11 , further comprising:
updating the page modification log to include the entry associated with the request, without triggering an exit of the virtual machine.
18 . A system comprising:
a physical processor of a first NUMA node; and
a physical memory of the first NUMA node, the physical memory including instructions that are executable by the physical processor for causing the physical processor to:
detect a request from a virtual machine to access a memory page, wherein the virtual machine is supported by a hypervisor of the first NUMA node, and wherein the request is detected by the physical processor rather than the hypervisor; and
in response to detecting the request, update a page modification log to include an entry indicating the request, wherein the hypervisor supporting the virtual machine is configured to:
monitor the page modification log over time to detect a change to the page modification log, wherein the change corresponds to the entry; and
based on detecting the change in the page modification log:
access the entry in the page modification log to indirectly detect the request; and
migrate the memory page from a second NUMA node to a destination NUMA node, the second NUMA node being separate from the first NUMA node.
19 . The system of claim 18 , wherein the hypervisor is further configured to:
determine a processor identifier associated with the request, the processor identifier being a unique identifier of an originating processor of the request;
access a mapping that correlates processor identifiers to NUMA node identifiers;
identify a correlation in the mapping between the processor identifier and a NUMA node identifier, wherein the NUMA node identifier corresponds to a particular NUMA node of the system, and wherein the correlation indicates that the particular NUMA node includes the originating processor; and
select the destination NUMA node for the memory page based on the particular NUMA node.
20 . The system of claim 18 , wherein the virtual machine includes a plurality of virtual processors, wherein a plurality of processor identifiers are configured to uniquely identify the plurality of virtual processors, and wherein the hypervisor is configured to:
generate a mapping that correlates the plurality of processor identifiers to a plurality of guest physical addresses (GPAs) associated with the memory page, wherein each virtual processor of the plurality of virtual processors is configured to request the memory page from a corresponding GPA of the plurality of GPAs; and
subsequent to generating the mapping:
detect the request;
determine a guest physical address associated with the request;
identify a correlation in the mapping between the guest physical address and a processor identifier, the correlation indicating that the request was issued by a virtual processor associated with the processor identifier;
identify a particular NUMA node that includes the virtual processor; and
select the destination NUMA node for the memory page based on the particular NUMA node.