Research On Spectrum Fusion Algorithm Based On Deep Reinforcement Learning And Federated Learning

With the development of communication technology,spectrum resource requirements are constantly increasing.However,under the traditional fixed spectrum allocation mechanism,limited spectrum resources cannot be fully utilized,and people’s communication requirements cannot be met.Cognitive radio technology,which enables dynamic spectrum access,has thus emerged to improve the utilization rate of spectrum resources and alleviate the shortage.The secondary user(SU)can dynamically access the idle spectrum for data transmission without affecting the communication of the primary user(PU).Deep reinforcement learning has been widely used in cognitive radio networks as an effective method to solve complex dynamic environment problems.In addition,as a new artificial intelligence technology,federated learning has also been attempted to be applied in wireless communication research.This paper uses deep reinforcement learning and federated learning to study spectrum fusion and spectrum access in cognitive radio networks.The main research work and innovations of this paper are as follows:Firstly,the technologies of cognitive radio networks are systematically organized.Then,the related theories and algorithms of deep reinforcement learning and federated learning are briefly introduced to provide theoretical and technical support for subsequent research.Secondly,this paper studies spectrum fusion technology based on spectrum sensing,spectrum sharing,and spectrum aggregation.With the premise of meeting the bandwidth requirement of the SU,spectrum fusion technology is applied to the spectrum access,thus improving spectrum efficiency.Thirdly,in response to the problem of low utilization of spectrum resources,this paper proposes a spectrum fusion scheme using the Maximum Entropy Based Actor-Critic algorithm to study the dynamic spectrum access of a single SU.The superiority of the proposed algorithm has been verified from two aspects:accumulative discounted reward and successful access rate.In addition,in the dynamic spectrum access of multiple SUs with different priorities,comprehensively considering aggregation capability,bandwidth requirement,and the priority of each SU,this paper proposes and implements a spectrum fusion scheme of multiple SUs by several distributed deep reinforcement learning algorithms.Simulation results show that the comprehensive performance of higher priority SU can be effectively guaranteed.Fourthly,the dynamic spectrum access problem of multiple SUs with the same priority is studied by proposing a corresponding spectrum fusion scheme.This paper explores a spectrum fusion algorithm based on federated learning and deep reinforcement learning,namely,the Federated Learning Based Maximum Entropy Multi-Agent Actor-Critic algorithm.The algorithm obtains data features from multiple users by model aggregation,thereby achieving the sharing of data features.The simulation results show that compared to the Maximum Entropy Based Multi-Agent Actor-Critic algorithm and the Deep Q Network algorithm,this algorithm performs better because the average successful access rate of multiple SUs is improved.