Goal-directed software-defined NUMA working set management
Initializing a software-defined server having software-defined NUMA domains includes, when booting a virtual environment defined by a set of hyper-kernels running on a plurality of physically interconnected computing nodes, accessing information associated with a software-defined NUMA domain configuration. It further includes, based at least in part on the accessed information, assigning software-defined NUMA domains to computing nodes. It further includes assigning virtualized resources to the software-defined NUMA domains. Handling a stalling event involving software-defined NUMA domains includes receiving an indication that a core or hyperthread which instantiates a virtual processor cannot access a virtualized resource needed by the core or hyperthread. It further includes determining, based at least in part on at least one of (1) a software-defined NUMA (Non Uniform Memory Access) domain to which the needed virtualized resource is assigned, (2) a software-defined NUMA domain to which the virtual processor is assigned, or (3) a software-defined NUMA domain associated with a computing node that has the core or hyperthread, whether to perform at least one of migrating the virtual processor or migrating the needed virtualized resource.
1. A computer system, comprising:
a plurality of physically interconnected computing nodes, wherein each of the plurality of physically interconnected computing nodes are separate physical devices;
a set of hyper-kernels, wherein:
each respective hyper-kernel of the set of hyper-kernels executes on a separate respective one of the plurality of physically interconnected computing nodes, and
the set of hyper-kernels collectively operate to form a software-defined server that comprises the aggregated resources of the plurality of physically interconnected nodes and on which a virtual machine spanning aggregated resources of the plurality of physically interconnected computing nodes executes; and
a guest operating system executing on the virtual machine spanning the plurality of physically interconnected computing nodes,
wherein:
when booting the virtual machine defined by the set of hyper-kernels executing on the plurality of interconnected computing nodes, information associated with a software-defined NUMA (Non Uniform Memory Access) domain configuration provided by at least one hyper-kernel is accessed by the guest operating system that is executing on the virtual machine, the software-defined NUMA configuration indicating a topology comprising a plurality of software-defined NUMA domains,
based at least in part on the accessed information, software-defined NUMA domains in the plurality of software-defined NUMA domains are assigned to computing nodes in the plurality of physically interconnected computing nodes, and
virtualized resources in a set of virtualized resources are assigned to the software-defined NUMA domains, wherein the set of virtualized resources is made available to the guest operating system that is run on the virtual machine.
2. The computer system of claim 1 wherein the software-defined NUMA configuration comprises a mapping of virtual processors and guest physical addresses to the plurality of software-defined NUMA domains, and wherein the mapping is presented to the guest operating system.
3. The computer system of claim 1 wherein the software-defined NUMA configuration comprises a set of relative access latencies between the plurality of software-defined NUMA domains, and wherein the set of relative latencies is presented to the guest operating system.
4. The computer system of claim 3 , wherein the set of relative access latencies is determined based at least in part on a relative mapping of the software-defined NUMA domains to the plurality of physically interconnected computing nodes.
5. The computer system of claim 1 wherein the at least one hyper-kernel maintains a mapping between the software-defined NUMA domains and the plurality of physically interconnected computing nodes.
6. The computer system of claim 1 wherein the at least one hyper-kernel maintains a first mapping of a virtualized resource of the set of virtualized resources and a current computing node of the plurality of physically interconnected computing nodes on which the virtualized resource currently resides.
7. The computer system of claim 6 wherein the at least one hyper-kernel maintains a second mapping of the software-defined NUMA domains to the current computing node.
8. The computer system of claim 1 wherein the assigning of the software-defined NUMA domains to the computing nodes is based at least in part on a user specification of a number of desired software-defined NUMA domains, and a number of computing nodes in the plurality of physically interconnected computing nodes.
9. The computer system of claim 1 wherein the software-defined NUMA configuration is determined based at least in part on information included in an emulated BIOS (Basic Input/Output System).
10. A method, comprising:
when booting a virtual machine defined by a set of hyper-kernels running on a plurality of physically interconnected computing nodes, accessing, by a guest operating system, information associated with a software-defined NUMA (Non Uniform Memory Access) domain configuration, the software-defined NUMA configuration indicating a topology comprising a plurality of software-defined NUMA domains, wherein:
each of the plurality of physically interconnected computing nodes are separate physical devices,
each respective hyper-kernel of the set of hyper-kernels executes on a separate respective one of the plurality of physically interconnected computing nodes,
the set of hyper-kernels collectively operate to form a software-defined server that comprises the aggregated resources of the plurality of physically interconnected nodes and on which a virtual machine spanning aggregated resources of the plurality of physically interconnected computing nodes executes; and
the guest operating system executes on the virtual machine spanning the plurality of physically interconnected computing nodes;
based at least in part on the accessed information, assigning software-defined NUMA domains in the plurality of software-defined NUMA domains to computing nodes in the plurality of physically interconnected computing nodes; and
assigning virtualized resources in a set of virtualized resources to the software-defined NUMA domains, wherein the set of virtualized resources is made available to the guest operating system.