Dynamic Spectrum Allocation Technology For User Satisfaction

Dynamic spectrum allocation,which can improve the spectrum efficiency of the static spectrum allocation and meet the explosive demand on spectrum resources in the wireless communications,has receive extensive attention in recent year.However,the existing works of dynamic spectrum allocation always utilized some quality of service(Qo S)metrics to evaluate their performance,such as throughput and system energy efficiency,which ignores the user’s subjective experience and thus cannot guarantee the global performance.Thus,considering the various user’s demands on spectrum resource and taking the quality of experience(Qo E)as user’s satisfaction metric for evaluating the effectiveness of dynamic spectrum allocation,reinforcement learning based the dynamic spectrum allocation methods are investigated in this paper.Specifically,the main contributions of this paper are summarized as follows:1.Aiming to satisfying the different performance requirements of multi-heterogeneous users in wireless networks,we firstly introduce Qo E to describe user satisfaction.Then,we summarize the main impact factors and propose the evaluation methods of Qo E.Considering the different Qo E requirements in the real-time and non-real-time scenario,we define the user satisfaction based on mean opinion score(MOS)and develop a MOS-based user’s satisfaction evaluation model,which can accurately depict the user’s subjective experience and thus contribute to the modeling of dynamic spectrum allocation.2.To address the spectrum allocation problem in the cognitive radio networks where multiple heterogeneous users have different Qo S requirements,we first develop a dynamic spectrum allocation model based on Qo E,and then a dynamic spectrum allocation method based on multi-agent reinforcement learning is proposed.Specifically,in order to improve the user’s satisfaction,the proposed method is evaluated by the Qo E instead of Qo S.In addition,multiple virtual agents are utilized to learn and interact with environment in a cooperative way,and thus the optimal spectrum allocation can be obtained by integrating their learning and spectrum decision results.Simulation results show that the proposed method can adapt to the dynamic environment and the multi-user scenario,and it can also effectively improve the Qo E and reduce the conflict probability between users.3.Under the dynamic spectrum allocation scenario where multiple heterogeneous users connect with each other with high density,we construct a Qo E-fairness tradeoff dynamic spectrum allocation model,and propose a dynamic spectrum allocation method based on adaptive deep Q-learning,which combines deep learning and reinforcement learning and breaks through the dimension limitations of traditional reinforcement learning.Then,we design a reward function to drive learning,and add the priority experience replay scheme to speed up the training speed of the network.Simulation results show that the proposed method can balance and improve the user’s Qo E and fairness under different device density scenarios,and reduce the time required for algorithm training with a good robustness.4.To verify the feasibility and effectiveness of the dynamic spectrum allocation method based on reinforcement learning,we build an experimental platform based on configurable frequency devices,and simulate the wireless transmission process for spectrum users with different demands.The experimental results show that the proposed reinforcement learning method can effectively achieve dynamic spectrum allocation and reduce the interference to authorized users such that the spectrum utilization can be effectively improved.