| With the rapid development of wireless communication technology and the continuous update of intelligent terminal technology for the Internet of Things,the number of wireless mobile users has increased exponentially in the past few years,resulting in increasingly serious scarcity of spectrum resources.At the same time,the traditional static spectrum allocation method has brought about The disadvantages of low spectrum utilization have become increasingly prominent.How to efficiently use limited spectrum resources and effectively improve spectrum utilization has become a top priority.As one of the key technologies of cognitive radio,dynamic spectrum access mainly studies how cognitive users can effectively access authorized spectrum in a dynamic environment.It is an important means to solve the current scarcity of spectrum and low spectrum utilization.This paper focuses on dynamic channel access in wireless networks,and studies the dynamic resource optimization problem based on deep reinforcement learning algorithms.Under different conditions of multiple channels for a single user and multiple channels for multiple users,efficient algorithm access is adopted.Into the model to improve the channel utilization of the system and maximize network benefits.The main research contents of this paper are as follows:(1)In the dynamic single-user multi-channel scenario,multiple channels are orthogonal to each other,and there is no information interaction,and this user always has data to transmit.Due to the limited channel status,this paper uses Markov chain to describe the correlation between channels and to describe the entire system.The user’s goal is to find a perception strategy that can obtain the largest expected cumulative discount reward in a limited time slot.This problem is expressed as a Markov decision process with unknown system dynamics,and two specific deep reinforcement learning methods are applied: Deep Q-Network and double Deep Q-Network,which overcomes the inability of traditional Q-learning methods to handle complex state actions space.The simulation results show that the network model based on the DQN algorithm can achieve near-optimal performance in this dynamic system.In order to improve the disadvantages of the DQN algorithm overestimating the action value under certain conditions,a network based on the DDQN algorithm is introduced to see from the convergence performance.The network based on DQN algorithm is 8 rounds faster.(2)Expanding from a single-user scenario to a multi-user scenario,considering the dynamic multi-channel access problem under multiple users,and considering the collision interference caused by multiple users accessing a channel at the same time,each user selects a channel to connect and transmit data,and judge whether the transmission is successful or not through a binary observation signal.The network aims to find a multi-user strategy that maximizes network utility without requiring online coordination or message exchanges between users.The paper uses a Dueling Deep Recurrent Q-Network algorithm to simulate,combined with long short-term memory network in order to maintain the internal state and summarize the observation results as the event progresses,which allows the network to use the history of the process to estimate real state.The simulation results show that the network performance of the Dueling Deep Recurrent Q-Network algorithm is superior to the traditional Dynamic Programming algorithm and the model based on a single deep Q-Network algorithm in terms of channel utilization and data rate. |
