Processing of communications signals using machine learning
One or more processors control processing of radio frequency (RF) signals using a machine-learning network. The one or more processors receive as input, to a radio communications apparatus, a first representation of an RF signal, which is processed using one or more radio stages, providing a second representation of the RF signal. Observations about, and metrics of, the second representation of the RF signal are obtained. Past observations and metrics are accessed from storage. Using the observations, metrics and past observations and metrics, parameters of a machine-learning network, which implements policies to process RF signals, are adjusted by controlling the radio stages. In response to the adjustments, actions performed by one or more controllers of the radio stages are updated. A representation of a subsequent input RF signal is processed using the radio stages that are controlled based on actions including the updated one or more actions.
1 . A method performed by one or more processors to control processing of radio frequency (RF) signals, the method comprising:
providing values representing a current state of a received RF signal as input to a machine learning network;
executing one or more policies of the machine learning network based on the values, to determine one or more actions to perform to control a plurality of radio stages of a radio communications apparatus, the plurality of radio stages comprising
a radio antenna, and
a tuner,
wherein the one or more policies are associated with estimations of results of signal processing using the one or more actions, and
wherein the machine learning network is configured to output an estimate of a result of a change in a center frequency of the tuner;
using a tuner controller of the radio communications apparatus, adjusting, in accordance with the estimate of the result of the change in the center frequency, the tuner to tune to a different center frequency;
receiving a first representation of an RF signal that is input to the radio communications apparatus;
processing the first representation of the RF signal using the radio antenna and the tuner in accordance with the different center frequency to which the tuner is tuned; and
based on the processing, obtaining, at an output of the plurality of radio stages, a second representation of the RF signal.
2 . The method of claim 1 , comprising adjusting the one or more policies of the machine learning network based on a measure of electromagnetic interference.
3 . The method of claim 1 , comprising adjusting the one or more policies of the machine learning network based on information about a signal emitter in an environment of the radio antenna.
4 . The method of claim 1 , wherein the machine learning network is configured to, based on the values representing the current state of the received RF signal, estimate respective results of increasing the center frequency and decreasing the center frequency.
5 . The method of claim 1 , comprising:
based on processing the first representation of the RF signal, obtaining, from the plurality of radio stages, output information describing an environment of the radio antenna;
determining one or more rewards based on the output information; and
storing the output information and the one or more rewards to a storage.
6 . The method of claim 5 , wherein the one or more rewards are based on one or more of the following indicated by the output information: a number of signals detected in the environment, a presence of electromagnetic interference in the environment, or a type of electromagnetic interference in the environment.
7 . The method of claim 1 , comprising:
obtaining one or more observations about the second representation of the RF signal;
determining one or more rewards based on the one or more observations; and
based on the one or more observations and the one or more rewards, adjusting the one or more policies of the machine learning network.
8 . The method of claim 7 , wherein adjusting the one or more policies comprises adjusting at least one of:
estimated rewards associated with actions used to control the plurality of radio stages,
estimated states associated with the actions used to control the plurality of radio stages, or
estimated rewards associated with states of the radio communications apparatus.
9 . The method of claim 7 , wherein adjusting the one or more policies comprises adjusting the one or more policies to achieve at least one target objective,
wherein the at least one target objective is based on at least one of user experience, throughput, latency, overhead, detection time for particular RF signals, identification of particular types of bursts for wideband extraction, resource utilization for multi-user capacity, signal detection robustness, signal detection speed, identification or reception of a radio signal or interference, mitigation of interference, bit error rate, power usage, a bandwidth requirement, processing complexity, or resource allocation.
10 . The method of claim 7 , wherein the machine learning network includes one of an artificial neural network (ANN), a deep dense neural network (DenseNN), or a convolutional neural network (ConvNN) comprising a series of parametric multiplications, additions, and non-linearities, and
wherein adjusting the one or more policies of the machine learning network includes
updating at least one of a connectivity in one or more layers of the ANN, or a weight of a connection in the one or more layers of the ANN, or
adjusting parameters of the machine learning network using reinforcement learning, Deep Q-Learning, Double Q-Learning, policy gradients, or an actor critic method.
11 . The method of claim 7 , wherein the one or more observations include at least one of channel response information, power spectrum information, cyclic feature information, time domain information, or spatial information.
12 . The method of claim 1 , wherein the values representing the current state of the received RF signal comprise frequency domain values of the received RF signal.
13 . The method of claim 1 , wherein the machine learning network is configured to estimate a result of an allocation of time-frequency-spatial spectrum resources to users in a communications system.
14 . The method of claim 1 , wherein the machine learning network is configured to estimate a result of at least one of an allocation of spectrum to users in a communications system, an allocation of time to users in the communications system, an allocation of frequency to users in the communications system, or an allocation of spatial slots to users in the communications system.
15 . The method of claim 1 , wherein the one or more policies includes a policy to detect presence of channel interference or distortion in communications channels in a communications system.
16 . The method of claim 1 , wherein the radio communications apparatus includes one of a base station in a cellular communications network or a cellular phone in the cellular communications network.
17 . A radio communications apparatus comprising:
a radio antenna;
a tuner
an tuner controller to control operations of the tuner; and
a computing device configured to perform operations comprising:
providing values representing a current state of a received RF signal as input to a machine learning network;
executing one or more policies of the machine learning network based on the values, to determine one or more actions to perform to control a plurality of radio stages of the radio communications apparatus, the plurality of radio stages comprising the radio antenna and the tuner,
wherein the one or more policies are associated with estimations of results of signal processing using the one or more actions, and
wherein the machine learning network is configured to output an estimate of a result of a change in a center frequency of the tuner;
using the tuner controller, adjusting, in accordance with the estimate of the result of the change in the center frequency, the tuner to tune to a different center frequency;
receiving a first representation of an RF signal that is input to the radio communications apparatus;
processing the first representation of the RF signal using the radio antenna and the tuner in accordance with the different center frequency to which the tuner is tuned; and
based on the processing, obtaining, at an output of the plurality of radio stages, a second representation of the RF signal.
18 . The radio communications apparatus of claim 17 , wherein the operations comprise adjusting the one or more policies based on a measure of electromagnetic interference.
19 . The radio communications apparatus of claim 17 , wherein the operations comprise adjusting the one or more policies comprises based on information about a signal emitter in an environment of the radio antenna.
20 . The radio communications apparatus of claim 17 , wherein the machine learning network is configured to, based on the values representing the current state of the received RF signal, estimate respective results of increasing the center frequency and decreasing the center frequency.