Wireless Resource Allocation For Multi-cell OFDM Communication System With Distributed Multi-agent Machine Learning Method

The traditional centralized wireless resource allocation implementation method is that the base stations collect the channel and demand information of their access users,and upload it to the upper-layer computing node.The upper-layer computing node obtains the optimal wireless resource allocation scheme for the entire network by solving an optimization problem,and then downloads it to the base stations for execution.When the number of base stations and users reaches a certain scale,it is difficult for the computing node to calculate the solution of the optimization problem within an effective time;in addition,the uploading and downloading of information also consumes unnecessary delay.The distributed multi-agent method can decompose the optimization problem into several sub-problems,and each sub-problem is calculated in parallel at the base stations.Such a distributed implementation method can avoid the dilemma of the centralized method.Aiming at the power and carrier allocation problem of multi-cell OFDM system,this paper adopts the realization method of distributed multi-agent,and adopts the method of machine learning in each agent,which can further improve the performance of resource allocation.The main work of the paper is as follows:Firstly,the multi-agent reinforcement learning method is used to solve the power allocation problem of multi-cell downlink OFDM system.In this paper,two new methods are proposed for the multi-cell downlink OFDM system power allocation interference coordination problem based on the DDPG reinforcement learning algorithm and the TD3 reinforcement learning algorithm.And two different learning architectures are designed,which are centralized training architecture and distributed training architecture respectively.These two training architectures do not require the base station to obtain global environmental state information,and the distributed training architecture only requires a small amount of information exchange between base stations,which overcomes many problems of traditional optimization solutions.Simulation results show that the proposed method has the advantages of easy implementation,strong robustness and high portability.Secondly,the distributed multi-agent reinforcement learning method is used to solve the problem of joint sub-carrier and power allocation in multi-cell downlink OFDM systems.For the case that the subcarrier allocation action set is a discrete set and the power allocation action space is continuous,this paper combines the DQN and DDPG reinforcement learning algorithms,and proposes a novel DQN-DDPG reinforcement learning algorithm to solve the joint optimization problem of subcarrier and power allocation.The simulation results show the effectiveness of the algorithm.Thirdly,the problem of resource allocation in multi-task federated learning is studied.This paper establishes a system model for the dual-task FL training process of wireless communication networks,and formulates an optimization problem that minimizes training time by jointly optimizing power allocation and user task selection.This paper simplifies the original complex non-convex problem by analyzing the FL convergence,and proposes an iterative algorithm to solve the problem.The simulation results show that the algorithm can effectively improve the performance.