Optimizing virtual machine scheduling on non-uniform cache access (NUCA) systems
Techniques for optimizing virtual machine (VM) scheduling on a non-uniform cache access (NUCA) system are provided. In one set of embodiments, a hypervisor of the NUCA system can partition the virtual CPUs of each VM running on the system into logical constructs referred to as last level cache (LLC) groups, where each LLC group is sized to match (or at least not exceed) the LLC domain size of the system. The hypervisor can then place/load balance the virtual CPUs of each VM on the system's cores in a manner that attempts to keep virtual CPUs which are part of the same LLC group within the same LLC domain, subject to various factors such as compute load, cache contention, and so on.
1. A method comprising:
partitioning, by a hypervisor of a computer system, a plurality of virtual central processing units (CPUs) of a virtual machine (VM) running on the computer system into one or more last level cache (LLC) groups, wherein the computer system includes at least one non-uniform cache access (NUCA) processor comprising a plurality of LLC domains, wherein each LLC domain includes K cores of the NUCA processor, and wherein a size of each LLC group is less than or equal to K; and
placing, by the hypervisor, each virtual CPU of the VM on a core of an LLC domain in the plurality of LLC domains based, at least in part, on the LLC group that the virtual CPU is a member of;
wherein a first virtual CPU in each LLC group is designated as a group leader and other virtual CPUs in the LLC group are designated as group followers, and
wherein the placing comprises, for each virtual CPU of the VM, determining whether the virtual CPU is a group leader or a group follower.
2. The method of claim 1 wherein the hypervisor attempts to place the plurality of virtual CPUs on a minimal number of LLC domains without exceeding a compute capacity of any single LLC domain.
3. The method of claim 1 wherein the computer system includes a plurality of NUCA processors that are part of a NUMA topology of the computer system, wherein the plurality of virtual CPUs are divided into one or more NUMA groups based on the NUMA topology, and wherein the hypervisor performs the partitioning in a manner that ensures the virtual CPUs in each LLC group are not members of different NUMA groups.
4. The method of claim 1 wherein the placing further comprises, if the virtual CPU is a group follower:
computing, for each of a plurality of candidate cores, a cost score indicating an execution efficiency cost for placing the virtual CPU on said each candidate core; and
placing the virtual CPU on a candidate core with the lowest cost score.
5. The method of claim 4 wherein the cost score is based on one or more CPU load factors and a leader affinity factor pertaining to current placement of the group leader of the virtual CPU's LLC group.
6. The method of claim 1 wherein the placing further comprises, if the virtual CPU is a group leader:
computing, for each of a plurality of candidate cores, a first cost score indicating an execution efficiency cost for placing the virtual CPU on said each candidate core and a second cost score indicating an execution efficiency cost for placing all virtual CPUs in the virtual CPU's LLC group on an LLC domain of said each candidate core;
computing an aggregated cost score based on the first and second cost scores; and
placing the virtual CPU on a candidate core with the lowest aggregated cost score.
7. A non-transitory computer readable storage medium having stored thereon program code executable by a hypervisor of a computer system, the program code embodying a method comprising:
partitioning a plurality of virtual central processing units (CPUs) of a virtual machine (VM) running on the computer system into one or more last level cache (LLC) groups, wherein the computer system includes at least one non-uniform cache access (NUCA) processor comprising a plurality of LLC domains, wherein each LLC domain includes K cores of the NUCA processor, and wherein a size of each LLC group is less than or equal to K; and
placing each virtual CPU of the VM on a core of an LLC domain in the plurality of LLC domains based, at least in part, on the LLC group that the virtual CPU is a member of;
wherein the computer system includes a plurality of NUCA processors that are part of a NUMA topology of the computer system, wherein the plurality of virtual CPUs is divided into one or more NUMA groups based on the NUMA topology, and wherein the hypervisor performs the partitioning in a manner that ensures the virtual CPUs in each LLC group are not members of different NUMA groups.
8. The non-transitory computer readable storage medium of claim 7 wherein the hypervisor attempts to place the plurality of virtual CPUs on a minimal number of LLC domains without exceeding a compute capacity of any single LLC domain.
9. The non-transitory computer readable storage medium of claim 7 wherein a first virtual CPU in each LLC group is designated as a group leader and other virtual CPUs in the LLC group are designated as group followers, and
wherein the placing comprises, for each virtual CPU of the VM, determining whether the virtual CPU is a group leader or a group follower.
10. The non-transitory computer readable storage medium of claim 9 wherein the placing further comprises, if the virtual CPU is a group follower:
computing, for each of a plurality of candidate cores, a cost score indicating an execution efficiency cost for placing the virtual CPU on said each candidate core; and
placing the virtual CPU on a candidate core with the lowest cost score.
11. The non-transitory computer readable storage medium of claim 10 wherein the cost score is based on one or more CPU load factors and a leader affinity factor pertaining to current placement of the group leader of the virtual CPU's LLC group.
12. The non-transitory computer readable storage medium of claim 9 wherein the placing further comprises, if the virtual CPU is a group leader:
computing, for each of a plurality of candidate cores, a first cost score indicating an execution efficiency cost for placing the virtual CPU on said each candidate core and a second cost score indicating an execution efficiency cost for placing all virtual CPUs in the virtual CPU's LLC group on an LLC domain of said each candidate core;
computing an aggregated cost score based on the first and second cost scores; and
placing the virtual CPU on a candidate core with the lowest aggregated cost score.
13. A computer system comprising:
a hypervisor;
at least one non-uniform cache access (NUCA) processor comprising a plurality of last level cache (LLC) domains; and
a non-transitory computer readable medium having stored thereon program code that causes the hypervisor to:
partition a plurality of virtual central processing units (CPUs) of a virtual machine (VM) running on the computer system into one or more LLC groups, wherein each LLC domain includes K cores of the NUCA processor, and wherein a size of each LLC group is less than or equal to K; and
place each virtual CPU of the VM on a core of an LLC domain in the plurality of LLC domains based, at least in part, on the LLC group that the virtual CPU is a member of;
wherein a first virtual CPU in each LLC group is designated as a group leader and other virtual CPUs in the LLC group are designated as group followers, and
wherein the placing comprises, for each virtual CPU of the VM, determining whether the virtual CPU is a group leader or a group follower.
14. The computer system of claim 13 wherein the hypervisor attempts to place the plurality of virtual CPUs on a minimal number of LLC domains without exceeding a compute capacity any single LLC domain.
15. The computer system of claim 13 wherein the computer system includes a plurality of NUCA processors that are part of a NUMA topology of the computer system, wherein the plurality of virtual CPUs are divided into one or more NUMA groups based on the NUMA topology, and wherein the hypervisor performs the partitioning in a manner that ensures the virtual CPUs in each LLC group are not members of different NUMA groups.
16. The computer system of claim 13 wherein a first virtual CPU in each LLC group is designated as a group leader and other virtual CPUs in the LLC group are designated as group followers, and
wherein the placing comprises, for each virtual CPU of the VM, determining whether the virtual CPU is a group leader or a group follower.
17. The computer system of claim 13 wherein the cost score is based on one or more CPU load factors and a leader affinity factor pertaining to current placement of the group leader of the virtual CPU's LLC group.
18. The computer system of claim 16 wherein the placing further comprises, if the virtual CPU is a group leader:
computing, for each of a plurality of candidate cores, a first cost score indicating an execution efficiency cost for placing the virtual CPU on said each candidate core and a second cost score indicating an execution efficiency cost for placing all virtual CPUs in the virtual CPU's LLC group on an LLC domain of said each candidate core;
computing an aggregated cost score based on the first and second cost scores; and
placing the virtual CPU on a candidate core with the lowest aggregated cost score.