Methods and systems for scheduling mmWave communications using reinforcement learning
A controller for scheduling mmWave communication is provided. The controller is programmed to identify a plurality of states, each of the plurality of states indicating status of mmWave communication links among a plurality of nodes, calculate an updated value for each of the plurality of states iteratively based on a value iteration algorithm and a previous value for each of the plurality of states until the updated value for each of the plurality of states converges, and select one of the plurality of states based on the converged values for the plurality of states.
1. A controller which is configured to:
receive an intent to communicate from one or more of a plurality of nodes;
identify a plurality of states based at least in part on the one or more intents to communicate, each of the plurality of states indicating status of mmWave communication links among the plurality of nodes;
calculate an updated value for each of the plurality of states iteratively based on a value iteration algorithm and a previous value for each of the plurality of states until the updated value for each of the plurality of states converges; and
select one of the plurality of states based on the converged values for the plurality of states, wherein:
the updated value for each of the plurality of states is calculated at least based on a reward value and a transition probability from a first state to a second state; and
the reward value is calculated based on a weight value related to a link to be added or removed and a number of conflicts due to an addition or a removal of the link.
2. The controller of claim 1 , wherein the controller is configured to:
broadcast information about the selected state over a V2X channel.
3. The controller of claim 1 , wherein the updated value for each of the plurality of states is calculated using Bellman equation.
4. The controller of claim 1 , wherein:
each of the mm Wave communication links is associated with a weight parameter; and
the updated value for each of the plurality of states is calculated further based on the weight parameter.
5. The controller of claim 1 , wherein the controller is configured to:
broadcast information about the selected state over a 5.9 GHz V2X channel.
6. The controller of claim 1 , wherein the plurality of nodes include a plurality of connected vehicles.
7. The controller of claim 1 , wherein the plurality of nodes include an edge server.
8. The controller of claim 1 , wherein one or more of the plurality of states include mm Wave communication links that conflict each other.
9. A method comprising:
receiving an intent to communicate from one or more of a plurality of nodes;
identifying a plurality of states based at least in part on the one or more intents to communicate, each of the plurality of states indicating status of mm Wave communication links among the plurality of nodes;
calculating an updated value for each of the plurality of states iteratively based on a value iteration algorithm and a previous value for each of the plurality of states until the updated value for each of the plurality of states converges; and
selecting one of the plurality of states based on the converged values for the plurality of states, wherein:
the updated value for each of the plurality of states is calculated at least based on a reward value and a transition probability from a first state to a second state; and
the reward value is calculated based on a weight value related to a link to be added or removed and a number of conflicts due to an addition or a removal of the link.
10. The method of claim 9 , further comprising:
broadcasting information about the selected state over a V2X channel.
11. The method of claim 9 , wherein the updated value for each of the plurality of states is calculated using Bellman equation.
12. The method of claim 9 , wherein:
each of the mm Wave communication links is associated with a weight parameter; and
the updated value for each of the plurality of states is calculated further based on the weight parameter.
13. A vehicle system comprising:
a controller which is configured to:
receive an intent to communicate from one or more of a plurality of nodes;
identify a plurality of states based at least in part on the one or more intents to communicate, each of the plurality of states indicating status of mmWave communication links among the plurality of nodes;
calculate an updated value for each of the plurality of states iteratively based on a value iteration algorithm and a previous value for each of the plurality of states until the updated value for each of the plurality of states converges;
select one of the plurality of states based on the converged values for the plurality of states; and
broadcast the selected state over a V2X channel, wherein:
the updated value for each of the plurality of states is calculated at least based on a reward value and a transition probability from a first state to a second state; and
the reward value is calculated based on a weight value related to a link to be added or removed and a number of conflicts due to an addition or a removal of the link.