Operation of sectorized communications from aerospace platforms using reinforcement learning
The disclosure provides a method of operating a communication network. The method includes receiving input data related to a state of the communication network and operation of the communication network. The method then includes determining a policy for the communication network based on the input data. The policy is a set of features for forming a plurality of communication links in the communication network over a time interval. The plurality of communication links provides one or more paths through the communication network. Determining the policy is based at least in part on utility values of previous policies. The utility values of previous policies are derived using simulation and/or real-world implementation of the previous policies. The communication network is then operated to implement the policy in the time interval.
1. A method of training a neural network, the method comprising:
receiving, by one or more processors, input data related to state information of a communication network that includes a plurality of nodes, a first node of the plurality of nodes being in motion relative to a second node of the plurality of nodes;
determining, by the one or more processors, a training policy based on input data related to operation of the communication network;
simulating, by the one or more processors, the training policy based on internal and external influences of the communication network; and
determining, by the one or more processors, a utility value of the training policy according to the simulation; and
storing the utility value and the training policy in order to enable the operation of the communication network based on the utility value, the training policy, and the neural network.
2. The method of claim 1 , wherein the neural network configured to receive a plurality of input features and process the input features based on the training policy in order to obtain a set of output features for a network configuration for a given time interval that allows the neural network to maximize a performance metric.
3. The method of claim 2 , wherein the performance metric relates to one or more of an amount of data transferred, a number of users reached, or a geographic area reached.
4. The method of claim 1 , wherein the training policy includes one or more of a set of communication links or characteristics of communication beams for each communication link that overall satisfy constraints presented by the input data for a first time period.
5. The method of claim 4 , wherein the set of communication links create one or more paths through the communication network according to the constraints.
6. The method of claim 1 , wherein the training policy includes characteristics of a communication beam for a communication link between a first communication device of a HAP terminal and a second communication device of a terrestrial communication terminal in a geographic area.
7. The method of claim 1 , wherein the training policy includes characteristics of a communication beam between a communication device of a HAP terminal and a client device in a geographic area.
8. The method of claim 7 , wherein the characteristics include one or more of a shape, power, direction, frequency, or channel designation of the communication beam over a first time interval.
9. The method of claim 1 , wherein the input data includes weather conditions or forecasts for atmospheric levels.
10. The method of claim 1 , wherein determining the utility value includes determining a predicted performance metric of the communication network in the simulation.
11. A system of training a neural network, the system comprising one or more processors configured to:
receive input data related to state information of a communication network that includes a plurality of nodes, a first node of the plurality of nodes being in motion relative to a second node of the plurality of nodes;
determine a training policy based on input data related to operation of the communication network;
simulate the training policy based on internal and external influences of the communication network; and
determine a utility value of the training policy according to the simulation; and
store the utility value and the training policy in order to enable the operation of the communication network based on the utility value, the training policy, and the neural network.
12. The system of claim 11 , wherein the neural network configured to receive a plurality of input features and process the input features based on the training policy in order to obtain a set of output features for a network configuration for a given time interval that allows the neural network to maximize a performance metric.
13. The system of claim 12 , wherein the performance metric relates to one or more of an amount of data transferred, a number of users reached, or a geographic area reached.
14. The system of claim 11 , wherein the training policy includes one or more of a set of communication links or characteristics of communication beams for each communication link that overall satisfy constraints presented by the input data for a first time period.
15. The system of claim 14 , wherein the set of communication links create one or more paths through the communication network according to the constraints.
16. The system of claim 11 , wherein the training policy includes characteristics of a communication beam for a communication link between a first communication device of a HAP terminal and a second communication device of a terrestrial communication terminal in a geographic area.
17. The system of claim 11 , wherein the training policy includes characteristics of a communication beam between a communication device of a HAP terminal and a client device in a geographic area.
18. The system of claim 17 , wherein the characteristics include one or more of a shape, power, direction, frequency, or channel designation of the communication beam over a first time interval.
19. The system of claim 11 , wherein the input data includes weather conditions or forecasts for atmospheric levels.
20. The system of claim 11 , wherein the one or more processors are further configured to determine the utility value includes determining a predicted performance metric of the communication network in the simulation.