Research On UAV Communication Transmission Optimization Strategy Based On Deep Reinforcement Learning

In recent years,integrating UAVs into cellular networks has been seen as a win-win solution for both the UAV-related industry and cellular network operators.On the one hand,UAVs equipped with small base stations can improve the quality of service for users on the ground and provide temporary network coverage.On the other hand,UAVs can obtain reliable low-latency and high-rate communication connections to meet their mission requirements with cellular infrastructure and advanced 5G-and-beyond technology.In this paper,the trajectory optimization problem of UAV integrated cellular network application is investigated to further enhance the overall performance of UAV communication system.The main research contents are as follows:(1)A multi-UAV assisted communication scenario was studied,in which the UAV base station needs to jointly design the trajectory to provide services for dynamically moving users and maximizing the total downlink transmission rate under the premise of considering the flight trajectory constraints(regional constraints)and communication constraints(SINR threshold).It is difficult for traditional optimization methods to solve this problem due to the unpredictability of user movement,the real-time performance of trajectory planning and the path gain to be unable to be represented by a closed solution since the presence of an irregularly distributed building.To solver the problem,this paper first reformulates the original path planning problem as Markov decision process by designing the state space,action space and reward function reasonably.Then,a multi-UAV dynamic trajectory optimization algorithm based on TD3 is proposed to solve the original problem effectively.Finally,the effectiveness of the proposed algorithm is verified by simulation experiments and comparison with the benchmark algorithm(2)The 3D path planning problem of UAV was studied,and a new trajectory planning framework for cellular-connected UAV assisted by radio map is designed,in which the UAV needs to minimize weighted flight time and expected outage duration considering the flight trajectory constraints(initial position,termination position).Because the path planning problem is NP-hard and the channel model cannot be expressed in closed form,it cannot be solved by the traditional convex optimization method.In order to solve this problem,the construction of 3D coverage map is firstly introduced,then the original path planning problem is transformed into Markov decision process,and a 3D path planning algorithm based on multi-step D3 QN is further proposed by using the constructed coverage map.Finally,the simulation experiment shows the effects of 3D antenna radiation mode of ground base station and no-fly zone on aerial coverage and UAV trajectory planning.The numerical results show that compared with 2D path with fixed height,3D path planning can greatly improve the overall performance.