Design And Simulation Of A Q-Network Enhanced Geographic Ad-Hoc Routing Protocol For The UAV Network

Unmanned aerial vehicles(UAVs)are widely used in several scenarios,such as military surveillance and public management.Meanwhile,the multi-UAV system is gradually showing its unique advantages.To construct a multi-UAV system,it’s necessary to build a highly reliable and low-latency UAV network,to provide communication services between different nodes.On the other hand,the rapid development of computing power in recent years make it possible to use machine learning methods,especially deep neural networks,in several applications such as image classification and speech recognition.Reinforcement learning,an essential area of the machine learning algorithm,also has evolved to be a good solution for many technieal problems.In this thesis,we try to optimize traditional geographic routing protocol for the ad-hoc net-work,to improve the network performance by using reinforcement learning.Firstly,we introduce the basic definition of reinforcement learning,such as Markov decision process and Temporal Dif-ference methods.Then,we analyze the critical technologies of three different routing protocols,including AODV,OLSR,and GPSR.Further,we propose QNGPSR,a Q-network enhanced ge-ographic routing protocol based on GPSR for the multi-UAV ad-hoc network.In QNGPSR,the neighbor topology information can be used to estimate the environment state and help reduce the usage of perimeter forwarding.The Q-network is used to predict the quality of next-hop selections.To evaluate the performances of OLSR,AODV,GPSR,and the proposed QNGPSR,we im-plement a network simulator based on SimPy,a process-based discrete-event simulation frame-work.The simulation results show that the proposed QNGPSR can obtain a lower end-to-end delay compared with the original GPSR in three different scenarios.When the node density is high,the latency of QNGPSR is almost equal to that of OLSR.At the same time,the delivery rate of QNGPSR still maintains the level of GPSR.When the network is busy,QNGPSR can provide better delay and packet delivery performance than other protocols.