Adaptive controller for online adaptation of resource allocation policies for iterative workloads using reinforcement learning
Techniques are provided for adaptive resource allocation for workloads. One method comprises obtaining a dynamic system model based on a relation between an amount of at least one resource for one or more iterative workloads and at least one predefined service metric; obtaining, from a resource allocation correction module, an instantaneous value of the at least one predefined service metric; and applying to a controller: (i) instantaneous parameters of the dynamic system model, and (ii) a difference between the instantaneous value of the at least one predefined service metric and a target value for the at least one predefined service metric, wherein the controller determines an adjustment to the amount of the at least one resource for the one or more iterative workloads. The obtained system model is optionally updated over time based on an amount of at least one resource added and the one or more predefined service metrics.
1. A method, comprising:
obtaining a dynamic system model that relates (i) an amount of at least one resource provided by an execution environment that executes one or more iterative workloads and (ii) at least one predefined service metric indicating a level of service provided by the execution environment for the one or more iterative workloads, wherein at least a first parameter and a second parameter of a plurality of parameters of the obtained dynamic system model are learned for a plurality of iterations of the one or more iterative workloads, wherein the learned first parameter represents an effect, on a value of the at least one predefined service metric, of an adjustment to the amount of the at least one resource provided by the execution environment and wherein the learned second parameter represents an obtained instantaneous value of the at least one predefined service metric;
and
applying to a controller: (i) the dynamic system model for a given iteration of the plurality of iterations of the one or more iterative workloads, and (ii) a difference between the instantaneous value for the given iteration of the at least one predefined service metric and a target value for the at least one predefined service metric, wherein the controller determines an adjustment, based at least in part on the difference and using the first parameter to modify one or more dynamics of a feedback loop that determines the difference, to the amount of the at least one resource to be applied in the execution environment for the one or more iterative workloads;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.
2. The method of claim 1 , wherein the obtained dynamic system model is one or more of: (a) derived from a relation between (i) an amount of at least one resource added in the execution environment in a given iteration and (ii) the predefined service level metric and (b) predefined based at least in part on the relation between the amount of the at least one resource added in the execution environment and the predefined service level metric.
3. The method of claim 1 , wherein the obtained dynamic system model is updated over time based on an amount of at least one resource added in the execution environment in a given iteration and the one or more predefined service metrics for the given iteration.
4. The method of claim 1 , wherein the iterative workload comprises a training of a Deep Neural Network.
5. The method of claim 1 , further comprising the step of applying the adjustment to the amount of at least one resource to a saturation model that bounds the difference between the instantaneous value of the at least one predefined service metric and one or more of the minimum and maximum values for the at least one predefined service metric.
6. The method of claim 1 , further comprising the step of applying the adjustment to the amount of at least one resource to an integrator that generates a cumulative resource allocation based on the adjustment to the amount of at least one resource for the one or more iterative workloads.
7. The method of claim 6 , further comprising the step of applying the cumulative resource allocation to a saturation model that bounds an applied resource allocation within an available amount of at least one resource.
8. The method of claim 1 , wherein the at least one resource comprises one or more of a number of processing cores, an amount of memory and an amount of network bandwidth.
9. A computer program product, comprising a non-transitory machine-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by at least one processing device perform the following steps:
obtaining a dynamic system model that relates (i) an amount of at least one resource provided by an execution environment that executes one or more iterative workloads and (ii) at least one predefined service metric indicating a level of service provided by the execution environment for the one or more iterative workloads, wherein at least a first parameter and a second parameter of a plurality of parameters of the obtained dynamic system model are learned for a plurality of iterations of the one or more iterative workloads, wherein the learned first parameter represents an effect, on a value of the at least one predefined service metric, of an adjustment to the amount of the at least one resource provided by the execution environment and wherein the learned second parameter represents an obtained instantaneous value of the at least one predefined service metric;
and
applying to a controller: (i) the dynamic system model for a given iteration of the plurality of iterations of the one or more iterative workloads, and (ii) a difference between the instantaneous value for the given iteration of the at least one predefined service metric and a target value for the at least one predefined service metric, wherein the controller determines an adjustment, based at least in part on the difference and using the first parameter to modify one or more dynamics of a feedback loop that determines the difference, to the amount of the at least one resource to be applied in the execution environment for the one or more iterative workloads.
10. The computer program product of claim 9 , wherein the obtained dynamic system model is one or more of: (a) derived from a relation between (i) an amount of at least one resource added in the execution environment in a given iteration and (ii) the predefined service level metric and (b) predefined based at least in part on the relation between the amount of the at least one resource added in the execution environment and the predefined service level metric.
11. The computer program product of claim 9 , wherein the obtained dynamic system model is updated over time based on an amount of at least one resource added in the execution environment in a given iteration and the one or more predefined service metrics for the given iteration.
12. The computer program product of claim 9 , wherein the iterative workload comprises a training of a Deep Neural Network.
13. The computer program product of claim 9 , further comprising the step of applying the adjustment to the amount of at least one resource to one or more of: (i) a saturation model that bounds the difference between the instantaneous value of the at least one predefined service metric and one or more of the minimum and maximum values for the at least one predefined service metric; and (ii) an integrator that generates a cumulative resource allocation based on the adjustment to the amount of at least one resource for the one or more iterative workloads.
14. The computer program product of claim 9 , wherein the at least one resource comprises one or more of a number of processing cores, an amount of memory and an amount of network bandwidth.
15. An apparatus, comprising:
a memory; and
at least one processing device, coupled to the memory, operative to implement the following steps:
obtaining a dynamic system model that relates (i) an amount of at least one resource provided by an execution environment that executes one or more iterative workloads and (ii) at least one predefined service metric indicating a level of service provided by the execution environment for the one or more iterative workloads, wherein at least a first parameter and a second parameter of a plurality of parameters of the obtained dynamic system model are learned for a plurality of iterations of the one or more iterative workloads, wherein the learned first parameter represents an effect, on a value of the at least one predefined service metric, of an adjustment to the amount of the at least one resource provided by the execution environment and wherein the learned second parameter represents an obtained instantaneous value of the at least one predefined service metric;
and
applying to a controller: (i) the dynamic system model for a given iteration of the plurality of iterations of the one or more iterative workloads, and (ii) a difference between the instantaneous value for the given iteration of the at least one predefined service metric and a target value for the at least one predefined service metric, wherein the controller determines an adjustment, based at least in part on the difference and using the first parameter to modify one or more dynamics of a feedback loop that determines the difference, to the amount of the at least one resource to be applied in the execution environment for the one or more iterative workloads.
16. The apparatus of claim 15 , wherein the obtained dynamic system model is one or more of: (a) derived from a relation between (i) an amount of at least one resource added in the execution environment in a given iteration and (ii) the predefined service level metric and (b) predefined based at least in part on the relation between the amount of the at least one resource added in the execution environment and the predefined service level metric.
17. The apparatus of claim 15 , wherein the obtained dynamic system model is updated over time based on an amount of at least one resource added in the execution environment in a given iteration and the one or more predefined service metrics for the given iteration.
18. The apparatus of claim 15 , further comprising the step of applying the adjustment to the amount of at least one resource to one or more of: (i) a saturation model that bounds the difference between the instantaneous value of the at least one predefined service metric and one or more of the minimum and maximum values for the at least one predefined service metric; and (ii) an integrator that generates a cumulative resource allocation based on the adjustment to the amount of at least one resource for the one or more iterative workloads.
19. The apparatus of claim 15 , wherein the at least one resource comprises one or more of a number of processing cores, an amount of memory and an amount of network bandwidth.
20. The apparatus of claim 15 , wherein the iterative workload comprises a training of a Deep Neural Network.