RIS-assisted UAV Downlink Communication Design Based On Deep Reinforcement Learning

With the rapid development of the sixth generation wireless communication technology,many potential devices are used in the field of wireless communication network.In order to improve the network communication capacity,improve the data transmission rate,enhance the signal coverage,improve the signal quality and other issues,there have been a lot of novel technologies,among which the Unmanned Aerial Vehicle(UAV)flexible deployment,convenience,high mobility and other characteristics,and can establish line-of-sight communication links with ground users,so it is widely used in the field of wireless communication.Uav can effectively expand the communication range,improve the network throughput,and avoid the signal loss caused by network load.However,in some remote mountainous areas or cities where natural disasters occur,due to the collapse of tall buildings and complex weather factors,the downlink communication channel between UAV and ground users cannot be obtained quickly and accurately,and the communication quality will be reduced.Therefore,looking for an auxiliary device to enhance the performance of UAV communication becomes a hot research direction.Recently,Reconfigurable Intelligent Surface(RIS),as a new technology in the sixth generation communication,has been paid attention by many scholars.It integrates a large number of low-cost passive reflection elements on the plane,intelligently reconfigure the wireless communication environment,so as to improve the performance of wireless transmission communication network.This paper attempts to use RIS as a repeater to assist the wireless downlink communication of UAV.Meanwhile,combining the respective advantages of UAV and RIS,it explores some potential possibilities and provides new ideas for the development of wireless communication technology in the future.The main work of this paper is as follows:(1)Considering that in some remote areas or communities where natural disasters occur,the outside world cannot communicate directly with users in the communities normally due to road blockage and building damage,the UAV is used to communicate with users on the ground,and RIS is deployed to assist communication.On the premise of meeting the requirements of UAV avoidance points and user acceptance of certain throughput,the problem of maximizing UAV propulsion energy efficiency was proposed,and the UAV flight path and RIS phase shift were jointly designed to maximize UAV propulsion energy efficiency.(2)In terms of the deployment location of RIS,this paper considers the deployment modes of RIS under two different circumstances.In addition,although Successive Convex Approximation(SCA)and other mathematical methods have been used to solve the Successive Convex approximation problems of UAV propulsion energy efficiency raised,However,the computing cost and complexity are high and cannot be deployed online.Considering these factors,this paper proposes a Deep Q-Network(DQN)algorithm to solve energy efficiency problems by transforming energy efficiency problems into Markov Decision Process(MDP).At the same time,the simulation results verify that RIS can effectively improve the system energy efficiency.(3)In terms of UAV 3D trajectory optimization,it is difficult to avoid the problem of overfitting combined with the proposed DQN algorithm.Thus,an advanced algorithm Dueling＿DDQN based on deep reinforcement learning is proposed to solve the problem of promoting energy efficiency.The simulation results verify that the Dueling＿DDQN algorithm can effectively reduce the UAV propulsion energy,improve the data transmission rate and propulsion energy efficiency.