Systems, devices and methods for scheduling wireless communications
Processing circuitry for a communication station configured to facilitate multi-user multiple-input multiple output (MU-MIMO) service. The processing circuitry can perform a multi-user selection for data transmission on a shared radio resource from a plurality of User Equipments (UEs). The processing circuitry selects one or more of the plurality of candidate UEs in time domain based on time-domain scheduling algorithm, obtain historical throughput data and input for each selected UE. The input includes a channel state indicator including a single-user channel quality indicator (SU-CQI), a precoding matrix indicator (PMI), rank indicator, and a channel state matrix. A trained reinforcement learning agent (RL agent) using the obtained input infers a rating score for each of the plurality of UEs. The processing circuitry schedules the one or more the UEs for transmission respectively on the plurality of radio resources based on the plurality of score ratings and allocate the plurality of radio resources.
1 . An apparatus for a communication station configured to provide multi-user multiple-input multiple output (MU-MIMO) service, the apparatus comprising:
processing circuitry, wherein to perform a multi-user selection for data transmission on a shared radio resource from a plurality of User Equipments (UEs), each of the UEs including a 5G new radio (NR), the processing circuitry is to:
select one or more of the plurality of candidate UEs in time domain based on time-domain scheduling algorithm,
obtain historical throughput data and input for each selected UE, the input comprising a channel state indicator including a single-user channel quality indicator (SU-CQI), a precoding matrix indicator (PMI), and rank indicator (RI), and a channel state matrix,
implement a trained reinforcement learning agent (RL agent) using the obtained input to infer a rating score for each of the plurality of UEs comprising the implemented trained RL agent to:
determine a correlation calculation between the UEs for each of a plurality of radio resources using the channel state matrices for each UE,
determine a multi-user channel quality indicator (MU-CQI) from the correlation calculation and the obtained SU-CQIs,
calculate a rating score for each of the plurality of UEs based on the correlation calculation, and the MU-CQI; and
schedule the one or more the UEs for transmission respectively on the plurality of radio resources based on the plurality of score ratings and allocate the plurality of radio resources.
2 . The apparatus of claim 1 ,
wherein to schedule the one or more UEs for transmission comprises the processing circuitry to check whether each of the plurality of UEs has a score rating greater or equal to a threshold and schedule each of the UEs having a score rating greater or equal to the threshold to the plurality of radio resources, and
wherein the processing circuitry is configured to, for each UE having a score rating that is less than the threshold, not schedule the UE having the score rating that is less than the threshold to a radio resource and set the score rating that is less than the threshold to zero.
3 . The apparatus of claim 1 ,
wherein the processing circuitry is further configured to reset the score rating of each scheduled UE after the resource allocation.
4 . The apparatus of claim 1 ,
wherein the processing circuitry is configured to determine the RL Agent award for each scheduling per Transmission Time Interval (TTI).
5 . The apparatus of claim 1 ,
wherein the trained RL agent implements a Deep Deterministic Policy Gradient (DDPG) type algorithm.
6 . The apparatus of claim 1 ,
wherein to determine the MU-CQI from the correlation calculation and the obtained SU-CQIs comprises to determine according to the formula:
M
U
CQI
=
1
M
*
1
1
su
CQI
+
max
ϑ
i
,
j
2
where M is a total number of selectable UEs,
where
max
ϑ
i
,
j
2
is the maximum value or correlation weight between a UE i and all the other UEs (UE j).
7 . The apparatus of claim 1 ,
wherein the apparatus comprises a wireless protocol stack.
8 . The apparatus of claim 1 ,
wherein the processing circuitry is configured to implement the RL Agent so as to infer the user selection in real-time.
9 . The apparatus of claim 1 ,
wherein to determine the correlation calculation between the UEs for each of a plurality of radio resources comprises the RL agent to perform one or more convolution operations on the channel matrix.
10 . The apparatus of claim 1 ,
wherein the processing circuitry is further configured to schedule the one or more the UEs for transmission together at one time.
11 . The apparatus of claim 1 ,
wherein to schedule the one or more UEs for transmission comprises to schedule the one or more UEs to a same radio resource based on the score ratings comprising to: determine a MU-MIMO gain for each UE after adding a subsequent UE to the same sub-band,
if the MU-MIMO gain decreases after the subsequent UE is added to a same sub-band remove or re-schedule the subsequently added UE.
12 . The apparatus of claim 1 ,
wherein the processing circuitry is further configured to determine a transmission throughput and fairness indicator based on transmission from the one or more scheduled UEs.
13 . The apparatus of claim 12 ,
wherein the processing circuitry is further configured to provide the transmission throughput and fairness indicator as feedback to the trained RL agent.
14 . The apparatus of claim 1 ,
wherein trained RL agent comprises a trained artificial neural network comprising: a plurality of interconnected neurons arranged in a plurality of layers, the plurality of interconnected neurons connected by a plurality of connections, the connections each including an associated weight, the weights determined by a training of the and the trained RL agent configured to provide outputs from the neurons indicating correlation calculation, the MU-CQI, and the score ratings for UEs.
15 . The apparatus of claim 1 ,
wherein the trained RL agent includes adjustable weights selected to determining in a direction of a maximum value of a reward
wherein the trained RL agent is trained with a data set comprising
a SU-CQIs for each of a group UEs,
a score rating for each of the group of UEs,
a channel matrix for each of the group of UEs,
a transmission throughput for each of the group of UEs,
wherein during training of the trained RL agent a correlation for each of a plurality of radio resources is determined from the channel matrix, and a MU-CQI is determined from the SU-CQIs, and
wherein the determined MU-CQI and throughput are multiplied by score ratings respectively producing a filtered MU-CQI and filtered throughput,
wherein the filtered MU-CQI and the filtered throughput are respectively input through a neural network, each neural network comprising an activation layer configured to normalize the filtered MU-CQI and the filtered throughput value, and
wherein weights a,b are respectively selected for the normalized MU-CQI and the normalized throughput so that the weights satisfy:
a
+
b
=
1
or
100
%
,
and
a
*
normalized
MU
-
CQI
+
b
*
normalized
throughput
=
reward
.
16 . The apparatus of claim 15 ,
wherein the correlation is determined by multiplying a processed version of the channel matrix by a transpose of the processed channel matrix,
wherein correlation matrix comprises elements representing the correlation weight between a UE i and all the other UEs, which is
max
ϑ
i
,
j
2
.
(UE j).
17 . The apparatus of claim 1 ,
wherein the communication station comprises a gNB node.
18 . The apparatus of claim 1 ,
wherein the base station comprises an electronic database comprising historical throughput data for UEs serviced by the base station, and
wherein the processing circuitry to obtain historical throughput data comprises the processing circuitry to obtain the historical throughput data from the electronic database.
19 . A method for performing user selection in a base station using multi-user multiple-input multiple output (MU-MIMO) service, the method comprising:
selecting one or more of the plurality of candidate UEs in time domain based on time-domain scheduling algorithm,
obtaining historical throughput data and input for each selected UE, the input comprising a channel state indicator including a single-user channel quality indicator (SU-CQI), a precoding matrix indicator (PMI), and rank indicator (RI), and a channel state matrix,
implementing a trained reinforcement learning agent (RL agent) using the obtained input to infer a rating score for each of the plurality of UEs comprising the implemented trained RL agent to:
determine a correlation calculation between the UEs for each of a plurality of radio resources using the channel state matrices for each UE,
determine a multi-user channel quality indicator (MU-CQI) from the correlation calculation and the obtained SU-CQIs,
calculate a rating score for each of the plurality of UEs based on the correlation calculation, and the MU-CQI; and
scheduling the one or more the UEs for transmission respectively on the plurality of radio resources based on the plurality of score ratings and allocate the plurality of radio resources.