Network nodes and methods performed therein for supporting handover of a wireless device
The present disclosure provides relates to methods and apparatus for handover optimization in a 5G context using reinforcement learning (RL). In contrast to the conventional handover methods, handovers between base-stations (BSs) are controlled using a centralized RL-based machine learning (ML) agent. This ML agent handles the radio measurement reports from the UEs and chooses appropriate handover actions in accordance with the RL machine learning framework to maximize a long-term utility.
1. A method implemented by a network node in a wireless communication network comprising multiple network nodes, the method comprising:
selecting, based on current radio environment conditions experienced by a wireless device, a radio environment context from a plurality of stored radio environment contexts indicative of radio conditions at respective locations in the wireless communication network, the selecting of the radio environment context being based on minimizing, using a Q-table comprising the plurality of stored radio environment contexts, a distance metric between the current radio environment conditions and each of the stored radio environment contexts in the Q-table, the radio environment context being a closest one of the stored radio environment contexts;
selecting a target network node for a handover using a mapping between the selected radio environment context and one or more candidate network nodes that are predicted, based on historical measurement data collected over time from a plurality of wireless devices, to provide the highest expected rewards in terms of a performance metric given the selected radio environment context; and
causing the handover of the wireless device to the target network node.
2. The method of claim 1 , wherein each of the stored radio environment contexts comprises a current serving cell and a set of one or more power measurement values associated with access beams transmitted by one or more neighboring network nodes.
3. The method of claim 2 , wherein each of the stored radio environment contexts further comprises at least one from a group consisting of a location of the wireless device, a traffic type associated with a data transmission to or from the wireless device, a quality of service requirement of the wireless device, a speed of the wireless device, and a direction of travel of the wireless device.
4. The method of claim 2 , wherein each of the stored radio environment contexts further comprises at least one selected from a group consisting of antenna configurations of the one or more neighboring network nodes, resource utilization of the one or more neighboring network nodes, and carrier frequencies of the one or more neighboring network nodes.
5. The method of claim 1 , further comprising determining the current radio environment conditions of the wireless device based on a measurement report received from the wireless device.
6. The method of claim 1 , wherein the one or more candidate network nodes are the neighboring network nodes predicted to provide highest link beam signal powers.
7. The method of claim 1 , wherein the highest expected rewards are determined by applying a machine learning to the historical measurement data collected from the plurality of wireless devices.
8. The method of claim 1 , further comprising sending an indication of the selected target network node to a serving base station or to the wireless device.
9. A method implemented by a network node to support handover of a wireless device from a source network node to a target network node, the method comprising:
determining a plurality of radio environment contexts indicative of radio conditions at respective locations within a coverage area of a wireless communication network, the radio environment contexts being configured for minimizing, using a Q-table comprising the plurality of stored radio environment contexts, of a distance metric between current radio environment conditions experienced by the wireless device and each of the radio environment contexts in the Q-table, the radio environment context being a closest one of the stored radio environment contexts;
generating a mapping between each of the radio environment contexts and one or more candidate network nodes that provide the highest expected rewards in terms of a performance metric for selecting one of the one or more candidate network nodes as a target network node for the handover given one of the radio environment contexts; and
transmitting the mapping to the source network node for use by the source network node in a handover of the wireless device to the target network node.
10. The method of claim 9 , wherein the performance metric comprises at least one of a group consisting of a data throughput, a signal quality, and a quality of service.
11. The method of claim 9 , wherein the mapping is generated by applying machine learning to historical measurement data collected over time from a plurality of wireless devices.
12. The method of claim 11 , wherein the mapping is generated by applying reinforcement learning to the historical measurement data.
13. The method of claim 12 , wherein the reinforcement learning is formulated by modeling the handover as a contextual multi-arm bandit.
14. The method of claim 9 , wherein each of the plurality of radio environment contexts comprises a current serving cell and a set of one or more power measurement values associated with access beams transmitted by one or more neighboring network nodes.
15. The method of claim 14 , wherein each of the plurality of radio environment contexts further comprises at least one from a group consisting of a location of the wireless device, a traffic type of a data transmission to or from the wireless device, a quality of service requirement of the wireless device, a speed of the wireless device, and a direction of travel of the wireless device.
16. The method of claim 14 , wherein each of the plurality of radio environment contexts further comprises at least one from a group consisting of antenna configurations of the one or more neighboring network nodes, resource utilization of the one or more neighboring network nodes, and carrier frequencies of the one or more neighboring network nodes.
17. The method of claim 9 , wherein the one or more candidate nodes are the neighboring network nodes predicted to provide highest link beam signal powers.
18. A network node for a wireless communication network comprising multiple network nodes, the network node being configured to:
select, based on current radio environment conditions experienced by a wireless device, a radio environment context from a plurality of stored radio environment contexts indicative of radio conditions at respective locations in the wireless communication network, the selection of the radio environment context being based on minimizing, using a Q-table comprising the plurality of stored radio environment contexts, a distance metric between the current radio environment conditions and each of the stored radio environment contexts in the Q-table, the radio environment context being a closest one of the stored radio environment contexts;
select a target network node for a handover using a mapping between the selected radio environment context and one or more candidate network nodes that are predicted, based on historical measurement data collected over time from a plurality of wireless devices, to provide the highest expected rewards in terms of a performance metric given the selected radio environment context; and
cause the handover of the wireless device to the target network node.
19. A network node for a wireless communication network comprising multiple network nodes, the network node being configured to:
determine a plurality of radio environment contexts indicative of radio conditions at respective locations within a coverage area of the wireless communication network, the radio environment contexts being configured for minimizing, using a Q-table comprising the plurality of stored radio environment contexts, of a distance metric between current radio environment conditions experienced by the wireless device and each of the radio environment contexts in the Q-table, the radio environment context being a closest one of the stored radio environment contexts;
generate a mapping between each of the radio environment contexts and one or more candidate network nodes that provide the highest expected rewards in terms of a performance metric for selecting one of the one or more candidate network nodes as a target network node for handover given one of the radio environment contexts; and
transmitting the mapping to a source network node for use by the source network node in the handover of a wireless device to the target network node.