Multi-Connection Heterogeneous Network Resource Management Technology Based On Deep Reinforcement Learning

With the continuous development of 4G and 5G technologies,the future network will have the characteristics of high dynamics,density variation and isomerization gradually.In the Future wireless network environment,there must be more than one type of network system for users,therefore,if user devices can aggregate different access network resource under the policies of the core network manager,the idle resources in wireless network will be fully used,the service quality of mobile user equipment will also be improved.But traditional single connection way for user equipment obviously unable to meet the needs of the user equipment aggregating a variety of network resources,the way of the multi-connection for users mainly divided into two kinds,one kind is the user equipment at the same time aggregates LTE cellular network resources and the WLAN resources based on 802.11 standard,the other is the user equipment aggregates 4G network port and 5G New Radio(NR)network port resources.Compared to single connection mode,mobile users access to multiple wireless network at the same time would make the system resources control more complex,most of the papers about the network resource control technology under the situations of user associating multiple wireless network exploring the optimal solution,will solve the optimization goal as NP-Hard problem,in order to find out the global optimal solutions or local optimal solution,the proposed algorithm computational complexity is much greater than the magnitude of polynomial computational complexity,and has no practical significance.Because of the self-adaptability of reinforcement learning,this paper explores the resource management and control technology that uses reinforcement learning method to solve the problem that users associate multiple networks,and designs the user access algorithm based on Deep Q-learning Network(DQN)and traffic allocation algorithm based on actor-critic(AC)framework respectively.Simulation results show that the performance of DQN based user selecting network algorithm is better than traditional user access algorithm,such as user nearest access algorithm.The performance of the traffic allocation algorithm based on AC framework is similar to a kind of local optimal solution for proportional fair user dual connection algorithm which based on matching,but its computational complexity is less than that algorithm.Finally,this paper introduces a heterogeneous network resource management architecture,and makes a simple simulation on the user multiple association algorithm using reinforcement learning method,and finds that when the user initiates themultimedia service with large capacity and high code rate,the service rate of the user has been significantly improved.The specific innovation points are introduced as follows:(1)the DQN can only deal with low dimensional discrete action space,but the model structure is simple and the computational complexity is lower,this paper apply it to the user associating network problem of coarse granularity and compared with the traditional user access algorithm,find that DQN based algorithm sacrifice a little calculation time delay as expense to improve the utilization ratio of the whole system,and have better balance.(2)Compare to the action space limitations of DQN,this paper further introduces AC framework,because AC framework use the deterministic strategies to direct output action value,it can tackle the problem of continuous or high dimensional discrete action space.Based on this feature,this paper apply it to fine-grained resource allocation problem,the simulation results find that performance of resource control technology based on AC framework with the polynomial computational complexity level can be very close to the performance of a kind of matching-based proportional fair user dual connection algorithm's local optimal solution.