Method for adaptive beam sweeping
The present invention incorporates reinforcement learning into a beam sweeping framework to select the appropriate set of beams to transmit reference signals in a predefined time interval for covering an angular region. More specifically, a network node starts a learning process to determine the most appropriate subset of beams from a large set of available beams (codebook) to communicate with an associated network node over a radio channel. The transmitter node acquires knowledge from its interaction with other nodes of the wireless network to perform beam sweeping with reduced signaling overhead and latency. More specifically, other advantages, the invention improves the beam management in higher carrier frequencies.
1. A method of adaptively selecting beams in a network node implemented in a wireless communication system, comprising:
determining, by the network node, a set of beams to transmit reference signals in a burst action set A={A # 1 , . . . , A #N} with N elements, where parameter N corresponds to a total number of the reference signals, wherein the determining is based on one of: random search, exhaustive search, iterative search and hierarchical search;
transmitting, from the network node, a set of reference signals using the set of beams contained in A to an associated node;
in response to the transmitting, receiving, from the associated node, measurements of a signal level of a radio link established between the network node and the associated node;
calculating, by the network node, based on the measurements of the received signal level and a correlation between beams of the set of beams, an associated reward (R) cost function, which comprises a set of associated weights W={W 1 , . . . , WX}, where parameter X corresponds to a number of network parameters considered in the associated reward (R) cost function;
updating, by the network node, a mapping of the burst action set A with the associated reward (R) cost function; and
retransmitting, by the network node, the set of reference signals using an updated burst action set A′, the updated burst action set A′ being obtained based on the updated mapping, and wherein the set of beams in the updated burst action set A′ is decorrelated beams.
2. The method as in claim 1 , wherein the determining of the set of beams comprises considering measurements of signal levels and a quality indication.
3. The method as in claim 1 , wherein the network node iteratively takes actions and measures impacts of the actions on the radio link based on the measurements of the signal level or a quality indication of the radio link.
4. The method as in claim 1 , wherein the set of associated weights are updated, by the network node, according to network goals.
5. The method as in claim 1 , wherein based on a negative impact being identified in network performance due to decisions taken by the network node, triggering a fallback to a predefined mechanism.
6. The method as in claim 5 , wherein the network performance is based on measurements of signal levels or a quality indication.
7. The method as in claim 1 , wherein a n-th element of the burst action set A is limited to a set of beams B={B # 1 , B # 2 , . . . , B #M}, determined according to an angular region to be covered in beam sweeping, where parameter M corresponds to a number of beams.
8. The method as in claim 7 , wherein parameters N and M are predefined by the network.
9. The method as in claim 1 , wherein the associated reward (R) cost function calculated by the network node is according to:
R
=
W
1
·
F
UNCORRELATION
(
A
)
+
W
2
·
F
KPI
(
A
)
where W 1 and W 2 are weight factors that determine impact of functions F UNCORRELATION and F KPI , respectively, F UNCORRELATION (A) measures a relationship among a set of selected beams determined by the burst action set, and F KPI (A) indicates a measurement of signal level or a quality indication of a radio channel between the network node and the associated node, where:
F
UNCORRELATION
=
1
F
CORR
(
A
)
;
F
KPI
(
A
)
=
KPI
(
A
)
KPI
TARGET
.
10. The method as in claim 9 , wherein a KPI is a channel quality indicator of the radio link between the network node and the associated node and KPI TARGET is a reference value of quality indication of a radio link defined for a network comprising the network node and the associated node according to requirements of provided services.
11. The method as in claim 1 , wherein the associated reward (R) cost function calculated by the network node measures an impact of correlation among beams and the quality of a radio link between the network node and the associated node:
R
=
W
3
·
F
CORR
(
A
,
B
1
TARGET
)
-
W
4
·
F
CORR
(
A
,
B
2
TARGET
)
where W 3 and W 4 are weight factors that determine the impact of a function F CORR on a value of the reward function; and F CORR is a correlation among a set of selected beams and a given beam B of interest; and
B 1 TARGET indicates the beam with an average measurement of signal level or quality indication equal to target value T 1 and B 2 TARGET indicates the beam with the average measurement of signal level or quality indication equal to target value T 2 .
12. The method as in claim 1 , wherein the set of reference signals is a synchronization signal block.
13. The method as in claim 1 , further comprising using, by the network node, a contextual multiarmed bandits (CMAB) approach, wherein the associated reward (R) cost function is:
R
=
R
+
RSRP
T
where T is a number of times that an action (A) was selected, and RSRP is a reference signal received power.
14. The method as in claim 13 , wherein the contextual multi-armed bandits (CMAB) has two distinct phases: exploration phase with probability P and exploitation phase with probability 1−P.
15. The method as in claim 14 , wherein in the exploration phase, a random action from the action (A) is selected.
16. The method as in claim 14 , wherein in the exploitation phase a best action based on a mapping among action values and context information is selected.