Research On Spectrum Allocation Based On Distributed Reinforcement Learning

Science and technology have seen unprecedented growth.many new communication devices have emerged,resulting in more and more communication devices need to be connected to the communication network to exchange information with each other.Hence,how to use the spectrum efficiently becomes increasingly important.Cognitive radio network is a relatively efficient solution,and its main task is to manage spectrum resources,which mainly includes two core issues: power control and channel allocation.Power control means that cognitive users can adjust their own transmission power to access the authorized frequency band without disturbing the communication of authorized users,so as to share spectrum resources with authorized users.Channel allocation refers to allocating some unused channels to users in need in a certain time slot,aiming at reducing conflicts among users and making full use of spectrum resources.The main content of this thesis is to combine multi-agent reinforcement learning with cognitive radio network,and propose an intelligent processing method for spectrum allocation.Compared with single-agent reinforcement learning,multi-agent reinforcement learning can learn more intelligent strategies through interaction in a higher-dimensional and dynamic environment,so it has natural advantages in solving multi-user spectrum allocation,which enables cognitive users to have certain intelligence and learn more superior control strategies.Corresponding solutions to the two core problems of spectrum resources are also proposed.Based on the above background,the main contents of this research as follow:1.Firstly,this thesis introduces the key technologies of cognitive radio and the core issues of its spectrum resource management,and focuses on the key technologies of spectrum allocation and the core reinforcement learning theory.2.Research and analyze the application of multi-agent deep reinforcement learning in multi-user power control,and propose a distributed power control strategy of double deep Q network combined with priority experience playback,which allows sub-users to adaptively adjust their own transmission power strategy in complex dynamic environment,improves the channel throughput and spectrum utilization,and enhances the overall network performance.3.Aiming at channel allocation,this thesis proposes a distributed channel allocation method based on deep reinforcement learning,which aims at a channel allocation strategy for multi-users.Based on the algorithm proposed in the second point above,considering the combination of algorithm model and channel allocation problem,a multi-user channel allocation algorithm with double-layer competitive deep Q network and long-term and shortterm memory is proposed,which enables sub-users to get the optimal allocation scheme when they successfully access the channel,reduce the collision between users,strengthen the coordination among users and improve the efficiency of the channel allocation.