Multi-timescale power control technologies
The present disclosure is related to power control mechanisms for workload processing systems, and in particular, multi-scale power control technologies that can be used to reduce the overhead of workload processing systems. The disclosed power control mechanisms operate on multiple timescales including a slow timescale and a fast timescale. Separate control loops (or governors) are used for the slow and fast timescales where each control loop includes its own trigger mechanisms and configurable operational policies. The operational policies for slow timescale control loop can be trained separately using various machine learning techniques while the operational policies for the fast timescale control loop can be simple and reactive heuristics.
1 . An apparatus for power management of a compute system, wherein the compute system operates according to a current operational policy, the apparatus comprising:
memory circuitry to store instructions; and
processor circuitry connected to the memory circuitry, and the processor circuitry is to execute the instructions to:
within a first power management control loop,
monitor first conditions including traffic load conditions of the compute system over a first duration at a first timescale, and
trigger a change from the current operational policy to a first operational policy based on the monitored first conditions; and
within a second power management control loop, trigger a change from the current operational policy to a second operational policy based on detected variations in second conditions including system load conditions of the compute system, the system load conditions including at least one of processor utilization, memory utilization, or pipeline pressure, over a second duration at a second timescale, wherein the second timescale is smaller than the first timescale.
2 . The apparatus of claim 1 , wherein the first timescale is a seconds-based timescale, a minute-based timescale, an hourly-based timescale, or a daily-based timescale; and the second timescale is a millisecond-based timescale, microsecond-based timescale, or nanosecond-based timescale.
3 . The apparatus of claim 1 , wherein the second conditions are based on one or more of processor core utilization of individual processor cores, processor time, processor core operating frequency of the individual processor cores, memory utilization, memory bandwidth, contested accesses, execution stalls, cache metrics for individual cache elements, assists metrics, floating point metrics, translation lookaside buffer (TLB) metrics, and energy analysis metrics.
4 . The apparatus of claim 1 , wherein the second condition is based on an internal measure of pressure or slack in a workload processing pipeline of the compute system, wherein the internal measure of pressure or slack is based on idle time measurements and branch prediction statistics.
5 . The apparatus of claim 1 , wherein the first operational policy includes first system settings for the compute system and the second operational policy includes second system settings for the compute system different than the first system settings.
6 . The apparatus of claim 5 , wherein:
the change from the current operational policy to the first operational policy includes reconfiguration of one or more hardware elements of the compute system to have the first system settings; and
the change from the current operational policy to the second operational policy includes reconfiguration of the one or more hardware elements to have the second system settings.
7 . The apparatus of claim 1 , wherein the processor circuitry is to execute the instructions to:
within the second power management control loop,
collect second condition measurements during a second timescale interval, and
determine whether to trigger the change from the current operational policy to the second operational policy based on the collected second condition measurements; and
within the first power management control loop,
collect first condition measurements when a time for a first timescale interval has begun, wherein the second timescale interval is shorter than the first timescale interval; and
determine whether to trigger the change from the current operational policy to the first operational policy based on the collected first condition measurements.
8 . The apparatus of claim 1 , wherein the compute system includes at least one processor, the at least one processor has a plurality of processor cores, and the current operational policy includes a set of frequencies that are currently set for respective processor cores of the plurality of processor cores.
9 . The apparatus of claim 8 , wherein:
the first operational policy includes setting a frequency of at least one processor core of the plurality of processor cores to be lower than a frequency of the at least one processor core when operating according to the current operational policy; and
the second operational policy includes setting a frequency of at least one processor core of the plurality of processor cores to be higher than a frequency of the at least one processor core when operating according to the current operational policy.
10 . The apparatus of claim 1 , wherein the first operational policy includes a configuration table, wherein individual entries in the configuration table includes a range of parameters for a respective hardware element of the compute system for a set of first conditions.
11 . The apparatus of claim 10 , wherein a machine learning model is used to learn the range of parameters for one or more hardware elements in the compute system for respective sets of the first conditions, wherein the machine learning model is a trained neural network or a reinforcement learning model.
12 . The apparatus of claim 11 , wherein the range of parameters includes one or more of a range of core frequencies, a range of uncore frequencies, a number of cache ways to be enabled or disabled, and a memory bandwidth.
13 . The apparatus of claim 12 , wherein the second operational policy includes a set of heuristics, wherein each heuristic of the set of heuristics includes a set of scaling factors to be used to adjust corresponding operational parameters of at least one hardware element of the compute system, wherein the set of scaling factors of each heuristic includes one or more of a core frequency scaling factor, an uncore frequency scaling factor, a number of cache ways to be enabled or disabled, and a memory bandwidth scaling factor.
14 . The apparatus of claim 1 , wherein the compute system is one of a network access node, a network element, a network appliance, an edge compute node in an edge computing network, a cloud compute node part of a cloud computing service, or an application server.
15 . A method of managing operation of a workload processing system, the method comprising:
operating a first control loop including:
monitoring a first condition including traffic load conditions of the workload processing system over a first timescale, and
triggering a change from a current operational policy of the workload processing system to a first operational policy of the workload processing system based on the monitored first condition; and
operating a second control loop including:
detecting variations in a second condition including system load conditions of the workload processing system, the system load conditions including at least one of processor utilization, memory utilization, or pipeline pressure, over a second timescale, wherein the second timescale is smaller than the first timescale, and
triggering a change from the current operational policy of the workload processing system to a second operational policy based on the detected variations in the second condition.
16 . The method of claim 15 , wherein the first timescale is a seconds-based timescale, a minute-based timescale, an hourly-base timescale, or a daily-based timescale, and the second timescale is a millisecond timescale, microsecond timescale, and/or nanosecond timescale.
17 . The method of claim 15 , wherein the first operational policy includes first system settings for the workload processing system and the second operational policy includes second system settings for the workload processing system different than the first system settings, and wherein:
the triggering the change from the current operational policy to the first operational policy includes reconfiguring one or more hardware elements to have the first system settings; and
the triggering the change from the current operational policy to the second operational policy includes reconfiguring the one or more hardware elements to have the second system settings.
18 . The method of claim 15 , wherein the method includes:
collecting second condition measurements during a second timescale interval;
determining whether to trigger the change from the current operational policy to the second operational policy based on the collected second condition measurements;
collecting first condition measurements when a time for a first timescale interval has begun, wherein the second timescale interval is shorter than the first timescale interval; and
determining whether to trigger the change from the current operational policy to the first operational policy based on the collected first condition measurements.
19 . The method of claim 15 , wherein:
the workload processing system includes at least one processor, the at least one processor has a plurality of processor cores, and the current operational policy includes a set of frequencies that are currently set for respective processor cores of the plurality of processor cores;
the first operational policy includes setting a frequency of at least one processor core of the plurality of processor cores to be lower than a frequency of the at least one processor core when operating according to the current operational policy; and
the second operational policy includes setting a frequency of at least one processor core of the plurality of processor cores to be higher than a frequency of the at least one processor core when operating according to the current operational policy.
20 . The method of claim 15 , wherein the method includes:
setting one or more processor cores of the workload processing system into respective first P-states in response to the change from the current operational policy to the first operational policy, wherein the respective first P-states are defined by the first operational policy, and wherein the respective first P-states have a performance capability set to be limited below a maximum performance capability; and
setting the one or more processor cores into respective second P-states in response to the change from the current operational policy to the second operational policy, wherein the respective second P-states are defined by the second operational policy, and wherein the respective second P-states have a performance capability set at a maximum performance capability.
21 . One or more non-transitory computer readable media (NTCRM) comprising instructions for managing operation of a workload processing system, wherein execution of the instructions is to cause a compute node to:
within a first control loop,
monitor first conditions including traffic load conditions of the workload processing system over a first timescale, and
trigger a change from a current operational policy of the workload processing system to a first operational policy based on the monitored first conditions; and
operating a second control loop including:
detecting variations in second conditions including system load conditions of the workload processing system, the system load conditions including at least one of processor utilization, memory utilization, or pipeline pressure, over a second timescale, wherein the second timescale is smaller than the first timescale, and
trigger a change from the current operational policy of the workload processing system to a second operational policy based on the detected variations in the second conditions.
22 . The one or more NTCRM of claim 21 , wherein execution of the instructions is to cause the compute node to:
measure a system slack or pressure of the workload processing system during a second timescale interval; and
determine whether to trigger the change to the second operational policy based on the measured system slack or pressure.
23 . The one or more NTCRM of claim 22 , wherein execution of the instructions is to cause the compute node to:
measure an input data rate to the workload processing system during a first timescale interval, wherein the second timescale interval is within the first timescale interval; and
determine whether to trigger the change to the first operational policy based on the measured input data rate.
24 . The one or more NTCRM of claim 21 , wherein execution of the instructions is to cause the compute node to:
set one or more processor cores of the workload processing system into respective first P-states in response to the change from the current operational policy to the first operational policy, wherein the respective first P-states are defined by the first operational policy, and wherein the respective first P-states have a performance capability set to be limited below a maximum performance capability; and
set the one or more processor cores into respective second P-states in response to the change from the current operational policy to the second operational policy, wherein the respective second P-states are defined by the second operational policy, and wherein the respective second P-states have a performance capability set at a maximum performance capability.