Granted Patent B2

US 11,042,410 · App. 16/380,311 · Granted Jun 22, 2021

Inventors: Yue Jin (Boulogne-Billancourt, FR); Dimitre Kostadinov (Paris, FR); Makram Bouzid (Orsay, FR); Armen Aghasaryan (Savigny sur Orge, FR)

Assignee: Nokia Solutions and Networks Oy

G06F9/5011G06F9/45558G06N20/00G06F2009/45579

Abstract

Claims (57)

1. An apparatus comprising:

at least one processor; and

at least one memory including computer program code,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least:

select a modification action having at least one resource variation parameter, wherein the resource variation parameter represents a resource quantity to be added or removed in a resource-controlled system;

update a value function of an adaptive learning agent as a function of one or more performance metrics measured after resources of the resource-controlled system have been modified by effecting the modification action, wherein the one or more performance metrics relate to the resource-controlled system, wherein the value function represents a desirability of the resource-controlled system as a function of one or more system states;

wherein selecting the modification action further causes the apparatus to at least:

select a first resource variation parameter, and

assess whether the modification action having the first resource variation parameter fulfills at least one safety criterion, and

substitute the first resource variation parameter by a second resource variation parameter having a finer scale resource quantity than the first resource variation parameter responsive to the safety criterion not being fulfilled;

modify the resources of the resource-controlled system by effecting the modification action prior to updating the value function of the adaptive learning agent;

determine a macro-modification action adjacent to the modification action by at least rounding the second resource variation parameter to an integer multiple of a predefined resource increment, wherein the value function is defined for the one or more system states resulting from one or more macro-modification actions, wherein the macro-modification action includes at least one resource variation parameter equal to the integer multiple of the predefined resource increment, and wherein the value function is updated for a system state which would have resulted from the macro-modification action adjacent to the modification action; and

reverse the modification action to further modify the resources of the resource-controlled system.

2. The apparatus of claim 1 , further configured to at least perform:

determine a reward function value for the modification action as a function of the performance metrics measured after the resources of the resource-controlled system have been modified by effecting the modification action; wherein the

reward function value for the macro-modification action is determined by at least extrapolating a reward function.

3. The apparatus of claim 2 , wherein the reward function is linearly extrapolated.

4. The apparatus of claim 1 , further configured to at least

compare a variance of the value function to a predefined threshold value to assess whether the safety criterion is fulfilled.

5. The apparatus of claim 1 , further configured to at least

compare the resource variation parameter to a predefined threshold value to assess whether the safety criterion is fulfilled.

6. The method of claim 1 , further configured to at least:

determine a number of prior occurrences of the modification action to assess whether the safety criterion is fulfilled.

7. The apparatus of claim 1 , wherein the first resource variation parameter is randomly selected.

8. The apparatus of claim 1 , wherein the first resource variation parameter is selected to maximize the value function in a current system state.

9. The apparatus of claim 1 , wherein selecting the modification action further comprises selecting a resource type in a set of resource types.

10. The apparatus of claim 1 , wherein the resource-controlled system comprises a data processing system, wherein the at least one resource variation parameter relates to a resource selected in the group comprising:

random access memory capacity, central processing unit processing power, central processing unit bandwidth, number of processor-cores, number of virtual machines, disk input/output bandwidth and network input/output bandwidth.

11. The apparatus of claim 10 , further configured to at least

send one or more resource modification instructions to a virtualization layer of the data processing system to effect the modification action.

12. A method comprising:

selecting a modification action having at least one resource variation parameter, wherein the resource variation parameter represents a resource quantity to be added or removed in the resource-controlled system;

updating a value function of an adaptive learning agent as a function of one or more performance metrics measured after the resources of the resource-controlled system have been modified by effecting the modification action, wherein the one or more performance metrics relate to the resource-controlled system, wherein the value function represents a desirability of the resource-controlled system as a function of one or more system states;

wherein selecting the modification action further comprises:

selecting a first resource variation parameter, and

assessing whether the modification action having the first resource variation parameter fulfills at least one safety criterion, and

substituting the first resource variation parameter by a second resource variation parameter having a finer scale resource quantity than the first resource variation parameter responsive to the safety criterion not being fulfilled;

modifying the resources of the resource-controlled system by effecting the modification action prior to updating the value function of the adaptive learning agent;

determining a macro-modification action adjacent to the modification action by at least rounding the second resource variation parameter to an integer multiple of a predefined resource increment, wherein the value function is defined for the one or more system states resulting from one or more macro-modification actions, wherein the macro-modification action includes at least one resource variation parameter equal to the integer multiple of the predefined resource increment, and wherein the value function is updated for a system state which would have resulted from the macro-modification action adjacent to the modification action; and

reversing the modification action to further modify the resources of the resource-controlled system.

13. The method of claim 12 , further comprising:

comparing a variance of the value function to a predefined threshold value to assess whether the safety criterion is fulfilled.

14. The method of claim 12 , further comprising:

comparing the resource variation parameter to a predefined threshold value to assess whether the safety criterion is fulfilled.

15. The method of claim 12 , further comprising:

determining a number of prior occurrences of the modification action to assess whether the safety criterion is fulfilled.

16. The method of claim 12 , wherein the first resource variation parameter is selected to maximize the value function in a current system state.

17. A non-transitory computer program comprising executable code for causing an apparatus to perform at least the following:

selecting a modification action having at least one resource variation parameter, wherein the resource variation parameter represents a resource quantity to be added or removed in a resource-controlled system;

updating a value function of an adaptive learning agent as a function of one or more performance metrics measured after the resources of the resource-controlled system have been modified by effecting the modification action, wherein the one or more performance metrics relate to the resource-controlled system, wherein the value function represents a desirability of the resource-controlled system as a function of one or more system states;

wherein selecting the modification action further comprises:

selecting a first resource variation parameter, and

assessing whether the modification action having the first resource variation parameter fulfills at least one safety criterion, and

substituting the first resource variation parameter by a second resource variation parameter having a finer scale resource quantity than the first resource variation parameter responsive to the safety criterion not being fulfilled;

modifying the resources of the resource-controlled system by effecting the modification action prior to updating the value function of the adaptive learning agent;

determining a macro-modification action adjacent to the modification action by at least rounding the second resource variation parameter to an integer multiple of a predefined resource increment, wherein the value function is defined for the one or more system states resulting from one or more macro-modification actions, wherein the macro-modification action includes at least one resource variation parameter equal to the integer multiple of the predefined resource increment, and wherein the value function is updated for a system state which would have resulted from the macro-modification action adjacent to the modification action; and

reversing the modification action to further modify the resources of the resource-controlled system by reversing the modification action.

Assignments (1)

Priority Claims (1)

Continuity (1)

Cited By (10)

US 12,210,984 US 12,217,197 US 12,254,427 US 12,400,154 US 12,412,120 US 12,412,131 US 12,412,132 US 12,524,820 US 12,547,991 US 12,651,275