Energy-efficient path planning system and method for internet of drones using reinforcement learning
A path planning system for an unmanned aerial vehicle in a network of unmanned aerial vehicles is disclosed. The system includes the unmanned aerial vehicles (UAVs). The system further includes a first processing circuitry configured with a particle swarm optimization component to offline generate paths for each of the UAVs by PSO to minimize path length and avoid static obstacles. The system further includes a second processing circuitry configured with a deep reinforcement learning (RL)-based planner component for each UAV, to perform real-time path planning to navigate the UAV through dynamic environmental conditions using a particular path generated by the PSO for the UAV as a consistent reference for the UAV. The system further includes a reward component to calculate a reward as part of the path planning by the deep RL-based planner component to determine potential paths and converge to an optimal path for the UAV.
1 . A path planning system for an unmanned aerial vehicle in a network of unmanned aerial vehicles, comprising:
a plurality of the unmanned aerial vehicles (UAVs);
a first processing circuitry configured with
a particle swarm optimization component configured to offline generate paths for each of the UAVs by particle swarm optimization (PSO) to minimize path length and avoid static obstacles; and
a second processing circuitry configured with, for each UAV of the plurality of UAVs,
a deep reinforcement learning (RL)-based planner component configured to perform real-time path planning to navigate the UAV through dynamic environmental conditions using a particular path generated by the PSO for the UAV as a consistent reference for the UAV, and
a reward component to calculate a reward as part of the path planning by the deep RL-based planner component to determine potential paths and converging to an optimal path for the UAV,
wherein the PSO is configured with an initialization stage in which chaos-based particles utilize a logistic map to obtain an initialization formation, and
wherein the PSO is configured to replace inactive particles with fresh particles, such that the RL-based planner converges towards a global optimum rather than getting stuck in a local optimum.
2 . The system of claim 1 , wherein the reward component is configured to use the path generated by the PSO for guidance in calculating the reward.
3 . The system of claim 2 , wherein the reward component, is further configured to
award a positive reward for following the PSO path, making this path preferable to others,
incur a negative reward as a penalty, when the UAV potentially collides with any obstacles or goes outside a boundary, and
award a positive reward as the UAV follows the PSO path and gets closer to a destination.
4 . The system of claim 1 , wherein the reward component is configured to assign a negative reward when the UAV reaches a free cell that is not part of the PSO.
5 . The system of claim 1 , further comprising an energy model configured to
calculate energy required by the UAV to reach a destination, and
add an additional amount of reward depending on the required energy.
6 . The system of claim 1 , wherein each UAV includes the second processing circuitry and the RL-based planner of the second processing circuitry is an actor-critic reinforcement learning model.
7 . The system of claim 1 , wherein each UAV includes the second processing circuitry and the RL-based planner of the second processing circuitry is a Q-learning model.
8 . The system of claim 1 , wherein the PSO is configured to adjust a mutation rate based on a particles' fitness values to balance exploitation and exploration of searching space.
9 . A path planning method for a plurality of unmanned aerial vehicles (UAVs), the method comprising:
generating, offline by particle swarm optimization (PSO) on first processing circuitry, paths for each of the UAVs to minimize path length and avoid static obstacles;
for each of the plurality of UAVs,
performing, by a deep reinforcement learning (RL)-based planner component on second processing circuitry, real-time path planning to navigate the UAV through dynamic environmental conditions using a particular path generated by the PSO for the UAV as a consistent reference for the UAV; and
calculating a reward to determine potential paths and converging to an optimal path for the UAV in the deep RL-based planner component,
the method further comprising
utilizing, by chaos-based particles, a logistic map to obtain an initialization UAV formation, and
replacing inactive particles with fresh particles, such that the RL-based planner converges towards a global optimum rather than getting stuck in a local optimum.
10 . The method of claim 9 , further comprising calculating the reward using the path generated by the PSO for guidance.
11 . The method of claim 10 , further comprising:
awarding a positive reward for following the PSO path, making this path preferable to others;
incurring a negative reward as a penalty, when the UAV potentially collides with any obstacles or goes outside a boundary; and
awarding a positive reward as the UAV follows the PSO path and gets closer to a destination.
12 . The method of claim 9 , further comprising assigning a negative reward when the UAV reaches a free cell that is not part of the PSO.
13 . The method of claim 9 , further comprising:
calculating energy required by the UAV to reach a destination; and
adding an additional amount of reward depending on the required energy.
14 . The method of claim 9 , further comprising adjusting a mutation rate based on a particles' fitness values to balance exploitation and exploration of searching space.
15 . A non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method for path planning for a plurality of unmanned aerial vehicles (UAV), the method comprising:
generating, offline by particle swarm optimization (PSO), paths for each of the UAVs to minimize path length and avoid static obstacles;
for each of the plurality of UAVs,
performing, by a deep reinforcement learning (RL)-based planner component, real-time path planning to navigate the UAV through dynamic environmental conditions using a particular path generated by the PSO for the UAV as a consistent reference for the UAV; and
calculating a reward to determine potential paths and converging to an optimal path for the UAV in the deep RL-based planner component,
the method further comprising
utilizing, by chaos-based particles, a logistic map to obtain an initialization UAV formation, and
replacing inactive particles with fresh particles, such that the RL-based planner converges towards a global optimum rather than getting stuck in a local optimum.
16 . The computer-readable storage medium of claim 15 , further comprising calculating the reward using the path generated by the PSO for guidance.