Path Planning Based On Reinforcement Learning In UAV-Assisted Communication

In recent years,with the development of technology,Unmanned Aerial Vehicles(UAVs)are developing towards miniaturization and inexpensive.With the advantages of easy deployment,controllability and mobility,UAVs,especially rotary-wing UAVs,have been widely used in civil and commercial fields,such as target tracking and detection,logistics,auxiliary communication and so on.Among them,UAV-assisted wireless communication is a new research hotspot.In UAV-assisted wireless communication,UAVs can be equipped with communication equipment to provide communication services for ground users,serve as relay nodes to establish connections for remote transceiver devices or serve as mobile sink nodes to collect data from ground wireless sensor networks.The UAV-assisted wireless communication studied in this paper mainly includes the following two application scenarios: UAVs serve as aerial base stations to provide communication coverage for ground users,and UAVs serve as mobile aggregation nodes to perform data collection tasks.In the first scenario,this paper considers a large-scale target area where basic communication facilities are damaged due to natural disasters and other reasons.UAVs equipped with communication equipment can make use of their mobility and height advantages to provide high quality and on-demand temporary communication services for ground users.Different from previous studies,this paper considers the fairness,introduces proportional fair scheduling algorithm to characterize the tradeoff between throughput and fairness,and proposes a distributed trajectory design algorithm based on multi-agent deep reinforcement learning algorithm to realize the distributed execution.The simulation results show that the proposed method is better than the baseline approach and is more suitable for the dynamic scene with high change and real-time decision making.In the scene of UAV-assisted data collection,UAV can collection data from sensing devices deployed in remote areas lacking communication coverage.Relying on its mobility,UAV can overcome the transmission distance limit of sensors,reduce energy consumption of sensing devices and extend the life of sensing network.This paper considers data collection in time-sensitive applications.The quality of service of such applications is closely related to the freshness of the data.This paper aims to to maximize the service quality based on the data freshness,and meanwhile,the endurance of the UAV is also considered.In this paper,the optimization problem is modeled as a semiMarkov decision process,and a path planning algorithm for single UAV based on deep Q network is proposed.The simulation results show that the proposed approach is better than the benchmark approach,and the balance between Qo S and safe electric quantity can be achieved by adjusting the parameters.In the future,UAVs will become a key component of wireless networks,and UAV assisted communication will be applied to more practical scenarios.In the future work,we plan to study the path planning problem of data collection by multi-UAVs cooperation.