Memory bandwidth throttling for virtual machines
Systems and methods are disclosed for throttling memory bandwidth accessed by virtual machines (VMs). A technique for dynamically throttling the virtual computer processing units (vCPUs) assigned to a VM (tenant) controls the memory access rate of the VM. When the memory is shared by multiple VMs in a cloud-computing environment, one VM increasing its memory access rate may cause another VM to suffer memory access starvation. This behavior violates the principle of VM isolation in cloud computing. In contrast to conventional systems, a software solution for dynamically throttling the vCPUs may be implemented within a hypervisor and is therefore portable across CPU families and doesn't require specialized server-class CPU capabilities or limit the system configuration.
1. A computer-implemented method, comprising:
calculating, according to a memory access metric, a first value corresponding to an execution of a first virtual machine (VM) of a plurality of VMs, the plurality of VMs being configured to share processing resources of a computing system according to timeslices allocated between the plurality of VMs;
detecting that the first value exceeds a threshold value;
determining, based on the first value, an idle time duration for which a first virtual CPU resource assigned to the first VM is placed into an idle state during a first set of the timeslices that are allocated to the first VM, wherein the first virtual CPU resource comprises at least one virtual CPU assigned to the first VM;
during execution of programmed instructions for the first VM, causing the first virtual CPU resource to be placed in the idle state for the idle time duration for at least one timeslice in the first set of the timeslices,
detecting, after the at least one timeslice, that the first value has decreased;
reducing the idle time duration for the first VM; and
causing the first virtual CPU resource to be placed in the idle state for the reduced idle time duration during a subsequent timeslice in the first set of the timeslices that is after the at least one timeslice.
2. The computer-implemented method of claim 1 , wherein the idle time duration is applied within each one of N timeslices within the first set of timeslices.
3. The computer-implemented method of claim 1 , wherein a duty cycle based on the idle time duration is applied across N timeslices included in the first set of timeslices.
4. The computer-implemented method of claim 1 , wherein the first timeslice is divided into M intervals and the idle time duration is applied within each one of the M intervals.
5. The computer-implemented method of claim 1 , wherein the memory access metric comprises a last level cache miss rate.
6. The computer-implemented method of claim 1 , wherein the memory access metric comprises at least one of: a memory access rate, an average memory access rate, or a count of memory accesses during a defined time interval.
7. The computer-implemented method of claim 1 , further comprising tracking performance data according to the memory access metric for each VM in the plurality of VMs.
8. The computer-implemented method of claim 1 , wherein performance data is sampled at least once each timeslice in the first set of the timeslices to produce the first value.
9. The computer-implemented method of claim 1 , further comprising:
detecting that the first value is equal or lower than the threshold value;
reducing the idle time duration for the first VM to zero; and
causing the first virtual CPU resource to be active during a subsequent timeslice in the first set of the timeslices.
10. The computer-implemented method of claim 1 , wherein the first virtual CPU resource comprises multiple virtual CPUs assigned to the first VM.
11. The computer-implemented method of claim 1 , further comprising:
detecting that a second value according to a second memory access metric for a second VM of the plurality of VMs exceeds a second threshold value;
determining, based on the second value, a second idle time duration for which a second virtual CPU resource assigned to the second VM is throttled by placing the second virtual CPU into the idle state during a second set of the timeslices that are allocated to the second VM, wherein the second virtual CPU resource comprises at least one virtual CPU assigned to the second VM; and
during execution of programmed instructions for the second VM, causing the second virtual CPU resource to be placed into the idle state for the second idle time duration for at least one timeslice in the second set of the timeslices.
12. The computer-implemented method of claim 11 , wherein the second threshold value is different compared with the threshold value.
13. The computer-implemented method of claim 1 , wherein the steps of detecting, determining, and throttling are executed using one or more computing resources of a cloud computing environment.
14. The computer-implemented method of claim 1 , wherein the step of determining the idle time duration is performed by a neural network.
15. The computer-implemented method of claim 1 , wherein the computing system is used for training, testing, or certifying a neural network employed in a machine, robot, or autonomous vehicle.
16. A system, comprising:
a computing system comprising processing resources shared by a plurality of virtual machines (VMs); and
a hypervisor that is configured to:
calculate, according to a memory access metric, a first value corresponding to an execution of a first VM of the plurality of VMs, the plurality of VMs being configured by the hypervisor to share the processing resources according to timeslices allocated between the plurality of VMs;
detect that the first value exceeds a threshold value;
determine, based on the first value, an idle time duration for which a first virtual CPU resource assigned to the first VM is placed into an idle state during a first set of the timeslices that are allocated to the first VM, wherein the first virtual CPU resource comprises at least one virtual CPU assigned to the first VM; and
during execution of code for the first VM, cause the first virtual CPU resource to be placed in the idle state for the idle time duration for at least one timeslice in the first set of the timeslices,
detecting, after the at least one timeslice, that the first value has decreased;
reducing the idle time duration for the first VM; and
causing the first virtual CPU resource to be placed in the idle state for the reduced idle time duration during a subsequent timeslice in the first set of the timeslices that is after the at least one timeslice.
17. The system of claim 16 , wherein the hypervisor is further configured to apply the idle time duration within each one of N timeslices within the first set of timeslices.
18. The system of claim 16 , wherein a duty cycle based on the idle time duration is applied across N timeslices included in the first set of timeslices.
19. A non-transitory computer-readable media storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
calculating, according to a memory access metric, a first value corresponding to an execution of a first virtual machine (VM) of a plurality of VMs, the plurality of VMs are configured to share processing resources of a computing system according to timeslices allocated between the plurality of VMs;
detecting that the first value exceeds a threshold value;
determining, based on the first value, an idle time duration for which a first virtual CPU resource assigned to the first VM is placed into an idle state during a first set of the timeslices that are allocated to the first VM, wherein the first virtual CPU resource comprises at least one virtual CPU assigned to the first VM; and
during execution of code for the first VM, causing the first virtual CPU resource to be placed in the idle state for the idle time duration for at least one timeslice in the first set of the timeslices,
detecting, after the at least one timeslice, that the first value has decreased;
reducing the idle time duration for the first VM; and
causing the first virtual CPU resource to be placed in the idle state for the reduced idle time duration during a subsequent timeslice in the first set of the timeslices that is after the at least one timeslice.