Deep Reinforcement Learning For UAV Communications

Unmanned aerial vehicle(UAV)becomes a key technology in wireless communications,due to it has some characteristics,such as controllable mobility,deployment flexibility,and rapid adaptability.This key technology has attracted widespread attention from researchers.This thesis first studies the problem of UAVs as air base stations,reducing interference to ground users and increasing the network throughput.Then the data collected from the sensors by the UAV-assisted wireless sensor network is studied to ensure the freshness of the collected data.Because the resource optimization problems related to the UAV communication are not only related to communication parameters,but also to the flight trajectory and energy of the UAV,they are often NP-hard,and traditional optimization methods are difficult to obtain the optimal solution to the problem.This thesis uses a method based on deep reinforcement learning(DRL).DRL does not need to directly solve complex optimization problems,instead,it interacts with the environment to continuously improve the policy to find the optimal solution to the problem.When the environment changes,DRL can quickly converge to the new optimal solution.The specific research contents of this thesis are as follows:First,the integration of the UAV base station and the terrestrial cellular network is studied.The UAV base station provides the UE with a second data connection,thereby increasing the network throughput.We optimize the association between the drone and the UEs to ensure that the utility function of the network throughput is maximized,and UEs are proportional fair.An association algorithm between UAVs and UEs based on deep reinforcement learning method is studied.Simulation results show that the algorithm based on deep reinforcement learning has the similar performance to the brute force search algorithm,but the time complexity of the algorithm based on deep reinforcement learning is small.And it can adapt to environmental changes,it can quickly converge to the optimal solution when the environment changes.Secondly,the freshness of data collected by the UAV-assisted IoT network is studied.In order to ensure the freshness of the data collected by the UAV,the flight trajectory of the UAV and the scheduling of the sensors are jointly optimized to minimize the weighted average AoI of the data collected by the UAV.Two scenarios are considered which the end position of the UAV is not specified and the end position of the UAV is specified.An algorithm for the UAV path planning and scheduling sensors based on deep reinforcement learning is studied.The influence of the specified end position of the UAV on the data collection of the UAV is analyzed through simulation,and we find that the UAV without the specified end position can collect data with a smaller weighted average AoI.The effects of the effective coverage radius of the sensor and the total energy of the UAV on the flight trajectory and scheduling sensor strategy of the UAV are further analyzed.Finally,the directions for the future research and problems that need to be solved are put forward,basing on the research of this thesis.