Research On UAV Network Deployment Method Based On Deep Reinforcement Learning

In sudden scenarios such as earthquake and flood where the existing network infrastructure suffers damage,or crowded scenarios such as concerts and sporting events,users will experience low quality and unstable mobile network services.It is of great importance to leverage Unmanned Aerial Vehicles(UAVs)carrying small-scale mobile base stations to form temporary UAV networks to improve the quality of mobile network services in special scenarios.For the methods of using UAV mounted base station to provide network services,most of the communication models are ideal,and the studies do not consider the demand changes brought by user mobility in the environment.For the studies on UAV path planning,most of them take the power of UAVs as a constraint,and the charging scheduling of UAVs is ignored.To address the above problems,this thesis proposes a deep reinforcement learning based UAV network deployment method,which is composed of two main parts.Firstly,considering the impact of factors such as multipath effect and shadow effect on signal propagation in the real environment,a 3D Convolutional DRL-based Chunk Selection Algorithm is proposed to select a chunk subset with the highest effective throughput for multi-UAVs in the dynamic environment as the area to be scanned actually.Secondly,considering the energy constraints of UAVs,an Energy-efficient DRL-based Chunk Search Algorithm is proposed to combine with charging scheduling to plan the flight paths for the UAVs to perform the chunks scanning task reasonably,so that UAVs can charge in time during the task of scanning chunk,and the total time of network services is maximized for all UAVs.The high quality of UAV network service is achieved by determining the best locations for UAVs while extending the service time of UAV networks.In this thesis,a small-scale prototype system is built for experiments and extensive large-scale simulation comparisons are carried out based on real data.The experimental results show that the chunk selection algorithm proposed in this thesis can achieve 95.35%effective throughput of the skyline algorithm with only 0.7%time overhead;the proposed chunk search algorithm extends the UAV network service time significantly and balances the workload of UAV base stations.