Research On Reinforcement Learning Based Transmission Scheme In UAV Millimeter Wave Communication System

The intelligent information society of the future is a highly digital and global data-driven society in which wireless communications play a very critical role.With the rapid development of smart terminals and the emergence of new applications,wireless data traffic is increasing dramatically,and the sixth generation(6G)mobile communication system will be an important technology to meet the new and rapidly growing demand for wireless communication.Therefore the ‘space-air-ground-sea integrated network’ has been proposed,which is gradually becoming one of the core technologies for6 G.Unmanned aerial vehicle(UAV)assisted wireless communication,as an important component of network,effectively complements the existing communication system.When UAV is used as an air base station,it can provide a reliable and economical wireless communication solution for various realistic scenarios.In addition,due to the shortage of spectrum resources,the application of millimeter-wave(mm Wave)to UAV and massive multiple input multiple output(MIMO)technology can create a new dynamic cellular network to provide high-capacity wireless communication services.In this paper,we focus on the UAV-assisted communication technology in the millimeter wave environment.In this thesis,the UAV-assisted mm Wave massive MIMO communication system and the channel model are established.In this system,the UAV equipped with multiple antennas is employed as a base station to serve ground user equipped with multiple antennas.The optimization performance of the proposed schemes is analyzed.The UAV-assisted communication system under line-of-sight(Lo S)link,where the UAV acts as an airborne base station to serve a stationary ground user,is first investigated.The communication process is modeled as a Markov decision-making process.The temporal-difference algorithm is used to optimize the trajectory of the UAV to improve the system’s performance.Secondly,the communication system of UAV base station serving a mobile ground user is studied,and the communication channel between the UAV and the user is modeled as a Rician channel.The precoding and trajectory of the UAV base station are optimized jointly by using the multi-arm bandit(MAB)and Q-learning algorithm.Compared with the existing precoding design for hovering flight,this optimization scheme can significantly improve the performance of the communication system.Finally,based on the above optimization scheme,the problem is extended to a static multi-user system,and the optimization problem is modeled as a maximum weight matching(MWM)problem in graph theory.Then,the combinatorial multi-arm bandit(CMAB)algorithm is introduced to solve the optimal precoding design in the joint optimization of this new scenario.This scheme can effectively enhance the performance of the communication system between the UAV base station and multiple ground users.