Trajectory Optimization Of Uav Base Stations Based On Reinforecement Learning

The unmanned aerial vehicle(UAV)is a kind of aircraft which is controlled by users or embedded system.UAV is widely used because of its small size,high flexibility and convenient deployment.The application of UAV in wireless communication is divided into two categories: on the one hand,UAV is considered as a new type of air user accessing cellular network for communication.On the other hand,UAV is used as an air communication platform,such as base station and relay,to assist ground users by providing data access.With the continuous reduction of UAV manufacturing cost and the miniaturization of communication equipment,it is now feasible to install small base station or relay equipment on UAV so that the UAV can assist the wireless communication system.Compared with traditional terrestrial base station,UAV as a base station has many advantages,such as providing a better channel condition,a more flexible deployment and so on.The trajectory optimization of UAV is an important issue in the design of UAV communication system.In practice,it is difficult to know exact location of mobile users when UAV is deployed.In addition,due to the limited onboard energy,the performance of UAV communication system is usually greatly reduced.Therefore,this paper studied and analyzed the trajectory optimization of UAV base station without accurate user location and proposed and solved the trajectory optimization problem of UAV base station that maximize the energy efficiency.In this paper,the communication system of UAV is studied firstly,and the trajectory optimization of UAV base station is modeled,then the reinforcement learning algorithm is studied.Finally,the trajectory optimization problem of UAV without accurate user position is solved based on grid method and Q-learning algorithm.On this basis,we proposed the trajectory optimization of UAV base station that maximize the energy efficiency.Due to the limitations of Q-learning algorithm,this paper used deep reinforcement learning algorithm to solve the problem.The simulation results show that the proposed algorithm can improve the energy efficiency of UAV base station.