Research On Multi-dimensional Resources Scheduling Algorithm For Wireless Networks Based On Deep Reinforcement Learning

The 6G network will introduce the ability of native intelligence and Artificial Intelligence(AI)will be fully integrated into the design of wireless network architecture to realize the wisdom-evolutionary and primitive-concise network,which has automation,self-optimization and autonomy capabilities.In this context,the definition of network resources will have a new connotation,including connection,data,computing,algorithm and other comprehensive resources.The allocation of network resources will break through the traditional way and require comprehensive cooperation of communication connection,communication bandwidth,computing power,algorithm and so on.Automatic analysis of specific services and joint allocation of multi-dimensional resources will be the key problem to be solved in network self-optimization and a typical scenario to be considered in the design of native-AI network architecture.The topic of this paper is selected from a joint research of Beijing University of Posts and Telecommunications-China Mobile Research lnstitute Joint Innovation Center.This paper discusses the design of nativeAI architecture in 6G network and uses Deep Reinforcement Learning(DRL)to solve the problem of multi-dimensional resources allocation including communication connection,communication bandwidth and computing power.The main work of this paper is as follows:(1)To solve the problem of cooperative scheduling of communication and computing resources in 6G wireless network,methods based on singleagent and multi-agent DRL are designed combining with native-AI network architecture.Firstly,a two-tier heterogeneous 6G computing network model consisted of Control Base Station(CBS)and Data Base Station(DBS)is built,in which communication and computing resources are jointly allocated.Then,a centralized single-agent resource scheduling scheme and a distributed multi-agent resource scheduling scheme are proposed and their deployments strategies in wireless network are planned.Furthermore,in multi-agent systems,Multi-Agent Proximal Policy Optimization(MAPPO)and Counterfactual Multi-Agent Policy Gradients(COMA)models are compared.Finally,the single-agent and multi-agent schemes are verified in the native-AI simulation platform.The results show that compared with the centralized single-agent Deep Deterministic Policy Gradient(DDPG),the distributed multi-agent has better convergence and global control ability in resources scheduling,and has better potential in optimizing network energy consumption and service delay.(2)To solve the task unloading problem of User Equipment(UE)in 6G wireless network,a task connection resources scheduling based on DDPG is designed in combination with native-Al architecture.Firstly,connection resources determined by task offloading,bandwidth resources and computing resources are co-scheduled in the 6G computing network model.Then,in order to further ensure the security and stability of network environment,a heuristic algorithm based on safe DRL is proposed and applied to the DDPG process to correct inaccurate actions.Finally,the proposed scheme is verified in the native-Al simulation platform.The results show that the introduction of heuristic algorithm improves the stability of model training and the efficiency of task offloading.The resources scheduling combined with DDPG and heuristic controls the UEs’selection of connection resources efficiently,and optimizes network consumption and service delay.In summary,based on the background of native-AI,this paper proposes resources scheduling schemes based on single-agent or multiagent DRL to adaptively allocate communication resources,computing resources and task connection resources in 6G network,which are verified and implemented in the native-AI simulation platform.The proposed multi-agent DRL scheme can effectively balance system energy consumption and service delay.At the same time,the proposed heuristic algorithm based on safe DRL can improve the model accuracy and training efficiency,and ensure the security and stability of network environment.