Managing power for serverless computing
Embodiments dynamically measure latency for a plurality of functions with a plurality of corresponding frequency levels within a serverless computing cluster; measure a transition latency from an idle state to an active state for the plurality of functions; determine whether a target response time to perform a service level objective (SLO) within the serverless computing cluster is going to be missed; dynamically reallocate at least one core and changing a frequency level across the plurality of functions by scaling down in response to a determination that the target response time to perform the SLO within the serverless computing cluster is going to be met; and dynamically reallocate the at least one core and changing the frequency level across the plurality of functions by scaling up in response to a determination that the target response time to perform the SLO within the serverless computing cluster is going to be missed.
1 . A method, comprising:
dynamically measuring, by a processor set, latency for a plurality of functions with a plurality of corresponding frequency levels within a serverless computing cluster;
measuring, by the processor set, a transition latency from an idle state to an active state for the plurality of functions;
determining, by the processor set, whether a target response time to perform a service level objective (SLO) within the serverless computing cluster is going to be missed;
dynamically reallocating, by the processor set, at least one core and changing a frequency level across the plurality of functions by scaling down in response to a determination that the target response time to perform the SLO within the serverless computing cluster is going to be met; and
dynamically reallocating, by the processor set, the at least one core and changing the frequency level across the plurality of functions by scaling up in response to a determination that the target response time to perform the SLO within the serverless computing cluster is going to be missed,
wherein the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling up comprises creating a new container instance for executing the plurality of functions on a different node than a node which includes the at least one core and degrading a performance of a function of the plurality of functions which has a lowest priority class of the plurality of functions, and
the determining whether the target response time to perform the SLO within the serverless computing cluster is going to be missed is based on execution time, queue length, number of cores, and the SLO with minimum power.
2 . The method of claim 1 , wherein the target response time comprises a summation of the transition latency for the functions and the measured latency for the functions.
3 . The method of claim 1 , wherein the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling up further comprises stealing the at least one core from a second function of a first node within the serverless computing cluster and reallocating the at least one core to a first function of the first node within the serverless computing cluster.
4 . The method of claim 1 , wherein the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling up further comprises stealing the at least one core from a second function of a second node within the serverless computing cluster and reallocating the at least one core to a first function of a first node within the serverless computing cluster.
5 . The method of claim 1 , wherein the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling up across the plurality of functions further comprises increasing the frequency level.
6 . The method of claim 1 , wherein the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling down comprises lending the at least one core from a first function of a first node within the serverless computing cluster and reallocating the at least one core to a second function of the first node within the serverless computing cluster.
7 . The method of claim 1 , wherein the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling down comprises lending the at least one core from a first function of a first node within the serverless computing cluster and reallocating the at least one core to a second function of a second node within the serverless computing cluster.
8 . The method of claim 1 , wherein the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling down comprises decreasing the frequency level.
9 . The method of claim 1 , wherein the determining whether the target response time to perform the SLO within the serverless computing cluster is going to be missed is based on a following formula: Execution time*((queue length/number of cores)+1)≤SLO, with min power*number of cores.
10 . A computer program product comprising one or more computer readable storage media having program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to:
measure power consumption needed to execute a function of a plurality of functions with a predetermined core frequency within a serverless computing cluster;
store the power consumption needed to execute the function with the predetermined core frequency in at least one lookup table;
determine that a power budget of the serverless computing cluster is going to be exceeded based on a number of cores and a next frequency level using the at least one lookup table; and
reduce a frequency in response to a determination that the power budget is going to be exceeded,
wherein the reducing the frequency comprises creating a new container instance for executing the function on a different node than a node which includes the predetermined core and degrading a performance of the function of the plurality of functions which has a lowest priority class of the plurality of functions, and
the determining whether the target response time to perform the SLO within the serverless computing cluster is going to be missed is based on execution time, queue length, number of cores, and the SLO with minimum power.
11 . The computer program product of claim 10 , further comprising reducing the number of cores in response to the determination that the power budget is going to be exceeded.
12 . The computer program product of claim 10 , further comprising turning off at least one core of the cores in response to the determination that the power budget is going to be exceeded.
13 . The computer program product of claim 12 , wherein the turning off at least one core comprises putting the at least one core in a deep sleep mode.
14 . The computer program product of claim 10 , further comprising determining whether a target response time to perform a service level objective (SLO) within the serverless computing cluster is going to be missed, the determining the target response time to perform the SLO is going to be missed is based on a tail latency for executing the function, and the tail latency represents a predetermined upper percentile of the target response time.
15 . The computer program product of claim 10 , wherein the next frequency level is a frequency level which ensures that the target response time to perform the SLO is going to be met, the next frequency level is different from the predetermined core frequency, and the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling down comprises decreasing the frequency level to provide just enough computing power to execute the plurality of functions and still meeting the SLO.
16 . A system comprising:
a processor set, one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to:
dynamically measure latency for a plurality of functions with a plurality of corresponding frequency levels within a serverless computing cluster;
measure a transition latency from an idle state to an active state for the plurality of functions;
determine whether a target response time to perform a service level objective (SLO) within the serverless computing cluster is going to be missed;
dynamically reallocate at least one core and changing a frequency level across the plurality of functions by scaling down in response to a determination that the target response time to perform the SLO within the serverless computing cluster is going to be met; and
dynamically reallocate the at least one core and change the frequency level across the plurality of functions by scaling up in response to a determination that the target response time to perform the SLO within the serverless computing cluster is going to be missed,
wherein the target response time comprises a summation of the transition latency for the functions and the measured latency for the functions,
the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling up comprises creating a new container instance for executing the plurality of functions on a different node than a node which includes the at least one core and degrading a performance of a function of the plurality of functions which has a lowest priority class of the plurality of functions, and
the determining whether the target response time to perform the SLO within the serverless computing cluster is going to be missed is based on execution time, queue length, number of cores, and the SLO with minimum power.
17 . The system of claim 16 , wherein the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling up further comprises stealing the at least one core from a second function of a first node within the serverless computing cluster and reallocating the at least one core to a first function of the first node within the serverless computing cluster.
18 . The system of claim 16 , wherein the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling down comprises lending the at least one core from a first function of a first node within the serverless computing cluster and reallocating the at least one core to a second function of the first node within the serverless computing cluster.
19 . The system of claim 16 , wherein the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling up across the plurality of functions further comprises increasing the frequency level.
20 . The system of claim 16 , wherein the dynamically reallocating the at least one core and changing the frequency level across the plurality of functions by scaling down comprises decreasing the frequency level to provide just enough computing power to execute the plurality of functions and still meeting the SLO.