Reinforcement learning of beam codebooks for millimeter wave and terahertz MIMO systems
View Patent ↗Reinforcement learning of beam codebooks for millimeter wave and terahertz multiple-input-multiple-output (MIMO) systems is provided. Millimeter wave (mmWave) and terahertz (THz) MIMO systems rely on predefined beamforming codebooks for both initial access and data transmission. These predefined codebooks, however, are commonly not optimized for specific environments, user distributions, and/or possible hardware impairments. To overcome these limitations, this disclosure develops a deep reinforcement learning framework that learns how to optimize the codebook beam patterns relying only on receive power measurements. The developed model learns how to adapt the beam patterns based on the surrounding environment, user distribution, hardware impairments, and array geometry. Further, this approach does not require any knowledge about the channel, radio frequency (RF) hardware, or user positions.
1 . A method for intelligently learning a beam codebook for multi-antenna wireless communications, the method comprising:
obtaining receive power measurements from a plurality of antennas; and
training the beam codebook using deep learning and the receive power measurements,
wherein the training of the beam codebook is achieved without knowledge of positions of wireless users within an environment accessed by a network node associated with the beam codebook.
2 . The method of claim 1 , further comprising beamforming wireless communications with a wireless device using the trained beam codebook.
3 . The method of claim 2 , further comprising initiating the wireless communications with the wireless device using the trained beam codebook.
4 . The method of claim 1 , wherein the training of the beam codebook uses the deep learning and the receive power measurements only.
5 . The method of claim 1 , wherein the training of the beam codebook is achieved without employing channel estimation.
6 . The method of claim 1 , wherein no knowledge of hardware details of communication circuitry that is employed at the network node associated with the beam codebook is used to train the beam codebook.
7 . The method of claim 1 , wherein the training of the beam codebook is achieved without knowledge of hardware details of communication circuitry that is employed at the network node associated with the beam codebook.
8 . A neural network for training of a beam codebook for multi-antenna wireless communications, the neural network comprising:
an actor network configured to predict one or more beam patterns for the beam codebook; and
a critic network configured to evaluate the one or more beam patterns predicted by the actor network based on receive power measurements of an environment,
wherein the training of the beam codebook is achieved without knowledge of positions of wireless users within an environment accessed by a network node associated with the beam codebook.
9 . The neural network of claim 8 , wherein the neural network comprises a Wolpertinger architecture.
10 . The neural network of claim 8 , wherein the training of the beam codebook uses deep learning and the receive power measurements only.
11 . The neural network of claim 8 , wherein the training of the beam codebook is achieved without employing channel estimation.
12 . The neural network of claim 8 , wherein the training of the beam codebook is achieved without knowledge of hardware details of communication circuitry that is employed at the network node associated with the beam codebook.
13 . The neural network of claim 8 , further comprising beamforming wireless communications with a wireless device using the trained beam codebook.
14 . A network node, comprising:
communication circuitry coupled to a plurality of antennas and configured to establish communications with a wireless device in an environment; and
a processing system configured to:
obtain receive power measurements from the plurality of antennas;
perform a machine learning-based analysis of the environment based on the receive power measurements; and
adapt the communications with the wireless device in accordance with the machine learning-based analysis of the environment,
wherein training of a beam codebook associated with the network node is achieved without knowledge of positions of wireless users of the wireless device within the environment accessed by the network node.
15 . The network node of claim 14 , wherein the communication circuitry comprises a radio frequency (RF) transceiver.
16 . The network node of claim 15 , wherein the RF transceiver is configured to communicate via at least one of a terahertz (THz) band or a millimeter wave (mm Wave) band.
17 . The network node of claim 14 , wherein the training of the beam codebook uses deep learning and the receive power measurements only.
18 . The network node of claim 14 , wherein the training of the beam codebook is achieved without employing channel estimation.
19 . The network node of claim 14 , wherein the training of the beam codebook is achieved without knowledge of hardware details of the communication circuitry that is employed at the network node associated with the beam codebook.
20 . The network node of claim 14 , wherein the processing system is further configured to:
beamform wireless communications with the wireless device using the trained beam codebook.