Research On Radio Resource Allocation Algorithm Of Multi-Cell Downlink OFDM System Based On Distributed Multi-Agents

The wireless resource allocation of mobile communication network is to allocate the three-dimensional wireless resources of "antenna,power and channel" for all accessed mobile users,so as to optimize some performance indexes of the system.The traditional wireless resource allocation algorithms are centralized,that is,all base stations report the user’s channel information and other requirements to the upper node.The upper node solves an optimization problem according to the reported information,and finally sends the solution of the optimization problem to the base station as a wireless resource allocation scheme for execution.This centralized processing has many disadvantages,among which the most fatal one is that with the increase of the scale of base stations and users,the computational complexity of solving the optimization problem increases exponentially,so that a single computing node can not bear such a high complexity of computing.In addition,the upload of information and the distribution of distribution scheme will also cause unnecessary delay.In order to avoid the above shortcomings,this paper takes the multi-cell downlink OFDM system as the background to study the wireless resource allocation scheme based on distributed multi-agents,that is,the centralized network-wide optimization problem is decomposed into several sub-problems.Calculation and information exchange during the calculation process can achieve the purpose of network-wide optimization.The main work is as follows:Firstly,based on maximizing the downlink sum rate,a distributed multi-agent computing method for joint allocation of user subcarriers and power in multi cell downlink OFDM system is studied.Firstly,according to the selection principle based on statistics,the users and all base stations in the system are optimally allocated;Then,the optimization problem model based on centralized solution is established;Because the centralized algorithm is too complex and will cause unnecessary waste of resources,a joint allocation algorithm of distributed multi-agent is proposed based on the idea of distributed solution;Due to the complexity of the optimization problem,the optimization problem is divided into two sub optimization problems,and an iterative algorithm of alternating optimization is proposed based on matching theory and dual subgradient iterative method to solve the above problems;Finally,the proposed distributed iterative algorithm is verified by simulation.The simulation results show that the algorithm can converge quickly,and has higher and speed than the other three baseline algorithms.Secondly,based on three different fairness indexes,the distributed multi-agent calculation method of user subcarrier and power joint allocation in multi cell downlink OFDM system is studied.Firstly,there are three different fairness indexes:subcarrier average allocation fairness,maximum and minimum fairness and maximum rate product fairness.Then,different optimization problems of wireless resource allocation are established for each index;Moreover,for different optimization problems,a distributed optimization algorithm to solve the corresponding optimization problem is proposed.Aiming at the optimization problem based on the fairness of subcarrier average allocation,a distributed algorithm is proposed.The optimal carrier allocation scheme is obtained based on the principle of subcarrier allocation to the best rate user,and the local optimal power allocation scheme is obtained based on the convex approximation of SCA.Then each agent performs the two schemes alternately and iteratively,and exchanges information among agents in a distributed way to optimize the scheme,Finally,the suboptimal solution of the problem is obtained;For the optimization problem based on maximum and minimum fairness,a distributed algorithm is proposed.Firstly,the original objective function is transformed into a new objective function that is easy to solve.Then,due to the complexity of the function,the problem is divided into two sub optimization problems.Based on SCA,the sub optimization problem of subcarrier allocation after variable relaxation and the sub optimization problem of power allocation are solved respectively,and then the agent iteratively executes the algorithm of the above two sub problems,Information exchange is carried out among agents in a distributed way to optimize the scheme until the overall algorithm converges,so as to obtain the suboptimal solution of the original optimization problem;For the optimization problem based on maximizing the fairness of rate product,a distributed algorithm is proposed.Similarly,due to the complexity and non convexity of the problem,the original problem is divided into two sub optimization problems.For the subcarrier allocation problem,a new subcarrier allocation algorithm is proposed in this chapter.For the power allocation problem,the SCA convex approximation is still used to obtain the solution of its power allocation,Then,the alternative iteration scheme is implemented by the agent,that is,the base station,in a distributed manner,and finally the suboptimal solution of the optimization problem is obtained.In addition,three distributed multi-agent algorithms to solve the optimization problem based on different fairness indicators are compared according to the unified measurement standard-fairness,so as to come to a conclusion that an algorithm can not only maximize the rate,but also maximize the fairness between users.Thirdly,based on maximizing the downlink sum rate,a distributed multi-agent reinforcement learning calculation method for j oint allocation of user subcarriers and power in multi cell downlink OFDM system is studied.Firstly,based on the idea of reinforcement learning,set the classical Quad,set the base station as the agent,and then let the agent and the environment try and error each other and interact with each other,so as to get the feedback from the environment to the agent.This feedback value is set to the total reachability and rate,that is,when only the information of the local cell is needed,the goal of maximizing the total reachability and rate of the system is realized through the above interaction process.Then,considering that the power allocation problem needs to get a continuous solution,referring to the basic principle of deep reinforcement learning algorithm ddpg algorithm,a Multi-Agent Reinforcement Learning Algorithm with centralized training and distributed execution is designed,and the corresponding system model and problem expression are created based on reinforcement learning;In addition,the proposed MultiAgent Reinforcement Learning Algorithm with centralized training and distributed execution is compared with the traditional distributed algorithm.Finally,the results are verified by simulation.The simulation results show that the agent distributed algorithm based on reinforcement learning has higher reachability and speed than the traditional distributed algorithm.