Efficient resource allocation for service level compliance
Various approaches to efficiently allocating and utilizing hardware resources in data centers while maintaining compliance with a service level objective (SLO) specified for a computational workload is translated into a hardware-level SLO to facilitate direct enforcement by the hardware processor, e.g., using a feedback control loop or model-based mapping of the hardware-level SLO to allocations of microarchitecture resources of the processor. In some embodiments, a computational model of the hardware behavior under resource contention is used to predict the application performance (e.g., as measured in terms of the hardware-level SLO) to be expected under certain contention scenarios. Scheduling of workloads among the compute nodes within the data center may be based on such predictions. In further embodiments, configurations of microservices are optimized to minimize hardware resources while meeting a specified performance goal.
1. A hardware processor comprising:
one or more processor cores and an uncore collectively supporting simultaneous execution of multiple workloads; and
within the uncore, an intelligent resource manager to control, during execution of a primary workload among the multiple workloads, allocation of one or more shared microarchitecture resources of the one or more processor cores and the uncore to optimize the allocation of the one or more shared microarchitecture resources subject to enforcing a hardware service level objective (SLO) received as part of an instruction set for the primary workload, the hardware SLO comprising a target value of an SLO metric derived from a performance guarantee associated with the primary workload pursuant to a service level agreement (SLA).
2. The hardware processor of claim 1 , wherein the SLO metric is a number of cycles per instruction (CPI) or instructions per cycle (IPC).
3. The hardware processor of claim 1 , wherein the one or more shared microarchitecture resources comprise at least one of last level cache (LLC) or main memory bandwidth.
4. The hardware processor of claim 1 , wherein the one or more shared microarchitecture resources comprise multiple shared microarchitecture resources controlled jointly to enforce the SLO.
5. The hardware processor of claim 1 , wherein the intelligent resource manager is to control the allocation of the one or more shared microarchitecture resources to optimize the allocation of the one or more shared microarchitecture resources, subject to enforcing the SLO, pursuant to an optimization policy received by the hardware processor.
6. The hardware processor of claim 1 , wherein the intelligent resource manager comprises control circuitry implementing a feedback control loop to control the allocation of the one more shared microarchitecture resources based on measurements of the SLO metric by dynamically adjusting the allocation of the one or more shared microarchitecture resources using the SLO metric as a process variable, the target value as a setpoint, and the allocation of the one or more shared microarchitecture resources as one or more control parameters.
7. The hardware processor of claim 6 , wherein the intelligent resource manager further comprises monitoring circuitry to compute the SLO metric based on one or more measured counters reported by the one or more processor cores and the uncore.
8. The hardware processor of claim 6 , wherein the feedback control loop comprises a proportional-integral-derivative (PID) controller.
9. The hardware processor of claim 8 , wherein the feedback control loop comprises a multiple-input multiple-output (MIMO) PID controller.
10. The hardware processor of claim 1 , wherein the intelligent resource manager comprises control circuitry to compute the resource allocations based on a machine-learning model.
11. The hardware processor of claim 1 , wherein the intelligent resource manager comprises control circuitry to compute a resource availability vector for achieving the target value of the SLO metric based on a workload signature model that represents a sensitivity of the SLO metric to the resource availability vector and a resource requirement vector associated with the primary workload, and to allocate the one or more shared microarchitecture resources according to the computed resource availability vector.
12. The hardware processor of claim 11 , wherein the control circuitry implements an inverse model of the workload signature model to compute the resource availability vector using the resource requirement vector and the target value of the SLO metric as inputs.
13. The hardware processor of claim 11 , wherein the control circuitry implements the workload signature model, and computes the resource availability vector iteratively by operating the workload signature model on input comprising the resource requirement vector and the resource availability vector to compute the SLO metric, and adjusting the resource availability vector until the computed SLO metric meets the target value.
14. The hardware processor of claim 11 , wherein the workload signature model comprises a machine-learned model.
15. The hardware processor of claim 14 , wherein the machine-learned model is based on training data comprising, for each of a plurality of collocation scenarios between primary and background workloads, associated measured resource availability and resource requirement vectors correlated with measured performance jitter values for the SLO metric.
16. A method comprising:
receiving, by a hardware processor comprising one or more processor cores and an uncore collectively supporting simultaneous execution of multiple workloads, an instruction set for a primary workload, the instruction set comprising a hardware service level objective (SLO) for the primary workload, the hardware SLO comprising a target value of an SLO metric derived from a performance guarantee associated with the primary workload pursuant to a service level agreement (SLA); and
controlling, by an intelligent resource manager within the uncore of the hardware processor, during execution of the primary workload, allocation of one or more shared microarchitecture resources of the hardware processor to optimize the allocation of the one or more shared microarchitecture resources subject to enforcing the SLO.
17. The method of claim 16 , wherein the SLO metric is a number of cycles per instruction (CPI) or instructions per cycle (IPC).
18. The method of claim 16 , further comprising receiving, by the hardware processor, an optimization policy, wherein the hardware processor controls the allocation of the one or more shared microarchitecture resources to optimize the allocation of the one or more shared microarchitecture resources pursuant to the optimization policy subject to enforcing the SLO.
19. The method of claim 16 , wherein controlling the allocation of the one more shared microarchitecture resources comprises measuring the SLO metric and dynamically adjusting the allocation of the one or more shared microarchitecture resources in a feedback control loop, using the measured SLO metric as a process variable, the target value as a setpoint, and the allocation of the one or more shared microarchitecture resources as one or more control parameters.
20. A system comprising:
a plurality of compute nodes, each comprising at least one hardware processor comprising one or more processor cores and an uncore collectively supporting simultaneous execution of multiple workloads, to execute one or more workloads associated with an application; and
a cluster orchestrator, comprising at least one hardware processor, to schedule the one or more workloads on one or more compute nodes of the plurality of compute nodes based at least in part on sufficient resource availability of one or more shared microarchitecture resources on the compute nodes to comply with hardware service level objectives (SLOs) derived for the workloads from an application-level SLO associated with the application pursuant to a service level agreement (SLA), each hardware SLO comprising a target value of a hardware SLO metric,
wherein the plurality of compute nodes are each to control, by an intelligent resource manager within the uncore of the respective at least one hardware processor, allocation of the one or more shared microarchitecture resources to any of the one or more workloads executed on the compute node to optimize the allocation of the one or more shared microarchitecture resources subject to meeting the hardware SLO.