Method for allocating transmission resources using reinforcement learning
The invention relates to a method for scheduling packets belonging to a plurality of data flow categories in a multi-access telecommunication system sharing a plurality of transmission resources. The method comprises at each transmission time interval, selecting, by a planner (CT, CF) a resource allocation plan and allocating the transmission resources to the data flows in accordance with the selected resource allocation plan. This selection is made by querying a look-up table (LUT) the content of which results from the implementation of a reinforcement learning and which makes it possible to identify, from the current state of the multi-access telecommunication system (s[t)]), the resource allocation plan to be selected, this plan being optimum to fulfil heterogeneous needs in terms of quality of service.
1. A method for determining a resource allocation for data flows belonging to a plurality of data flow categories in a multi-access telecommunication system sharing a plurality of transmission resources, said method comprising, for a state of the multi-access telecommunication system, a step of determining, by iterations of a reinforcement learning, an optimum resource allocation plan maximising a reward sum, each iteration of the reinforcement learning comprising:
allocating the transmission resources to the data flows according to an iteration resource allocation plan;
transmitting the data flows by way of the allocated transmission resources;
for each of the data flow categories, calculating at least one transmission performance indicator of each of the data flows of the data flow category, and comparing, for each packet of the data flow category, the calculated at least one transmission performance indicator with a threshold representative of a quality of service requirement relating to the at least one transmission performance indicator for the data flow category, and
determining a reward as a function of the result of said comparing for each of the data flows of each of the categories.
2. The method according to claim 1 , wherein the iteration resource allocation plan comprises a hierarchisation rule and at least one first and one second scheduling rule, and wherein allocating the transmission resources to the data flows according to the iteration resource allocation comprises two steps consisting in:
a scheduling in the time domain made by a time classifier configured to hierarchise the data flow categories according to the hierarchisation rule into at least one priority category and one secondary category; and
a scheduling in the frequency domain made by a frequency classifier configured to:
schedule the data flows of the priority category according to the first scheduling rule and allocate the transmission resources to the data flows of the priority category as they are scheduled; and
in case of remaining transmission resources, schedule the data flows of the secondary category according to the second scheduling rule and allocate the remaining transmission resources to the data flows of the secondary category as they are scheduled.
3. The method according to claim 2 , wherein they reward determined at each iteration of the reinforcement learning is the sum
∑
p
=
1
P
r
p
(
a
CT
,
u
p
)
,
with
r
p
(
a
CT
,
u
p
)
=
∑
i
p
=
1
I
p
∑
o
=
1
N
O
P
(
1
-
x
_
p
,
o
-
x
p
,
i
p
,
o
)
,
where P designates the number of data flow categories, a cT the hierarchisation rule, u p the scheduling rule applied to the data flows of the data flow category p, I p the number of data flows of the data flow category p, N Op the number of transmission performance indicators of the data flow category p, x p,o the threshold representative of a quality of service requirement relating to the transmission performance indicator o for the data flow category p x p,i p,o and the transmission performance indicator o of the data flow i p .
4. The method according to claim 1 , wherein each iteration of the reinforcement learning comprises:
according to a probability P a , an exploitation selection consisting in selecting as the optimum resource allocation plan the iteration resource allocation plan maximising the reward sum at this stage of iterations; and
according to a probability P a* , an exploration selection consisting in selecting a resource allocation plan a resource allocation plan different from the iteration resource allocation plan maximising the reward sum at this stage of iterations.
5. The method according to claim 1 , wherein the at least one transmission performance indicator is one from a transmission delay, a transmission rate and a transmission loss rate.
6. The method according to claim 1 , further comprising, at a transmission time interval, allocating the transmission resources in accordance with the optimum resource allocation plan determined for [[the]] a current state of the multi-access telecommunication system.
7. A non-transitory computer program product comprising program code instructions for executing the method according to claim 1 when said program is executed by a computer.