Research On Resource Allocation Method Of Intelligent Reflecting Surface Assisted OFDM Communication System Based On Deep Reinforcement Learning

Orthogonal frequency division multiplexing(OFDM)is widely used in many communication systems such as LTE and the fifth-generation wireless communication networks.By utilizing orthogonal sub-carriers,it can achieve high-speed and robust information transmission,as well as effectively avoid inter-channel interference.However,with the development of wireless communication networks,the severe wireless channel fading makes it a big challenge for OFDM systems to meet the demand of higher rate and lower delay.Therefore,there is an urgent need for a new technology to support OFDM system to break through the existing performance bottleneck.Recently,intelligent reflecting surface(IRS)has been regarded as a green and economic solution.The IRS can freely adjust the phase and amplitude of the reflection unit,realize the adjustment of the direction and amplitude of the incident signal,change the wireless communication environment,and further improve the performance of the communication system.The resource allocation algorithms based on optimization techniques are confronted with the intractable dimension curse problem when the increase of the problem complexity and the expansion of the communication network scale.Recently,deep reinforcement learning(DRL)has attracted wide attention as a superior artificial intelligence technology.Due to its outstanding advantages of fast and real-time processing of complex non-convex problems,which is very suitable for solving large-scale wireless communication resource optimization problems.This thesis focus on the intelligent resource allocation design of IRS-assisted OFDM system based on DRL,the joint design of IRS-assisted spectrum sharing OFDM system based on hybrid DRL and the physical layer security design of IRS-assisted OFDM system based on distributed DRL.This thesis studies the intelligent resource allocation scenario of IRS-assisted OFDM system.The downlink users sum rate is maximized by jointly optimizing the sub-carrier allocation,the transmit beamforming of the base station and the phase shift of the IRS.An intelligent resource allocation scheme based on combining deep Q networks(DQN)and deep deterministic policy-gradient(DDPG)is proposed to tackle the problem of the hybrid action space optimization.For the discrete sub-carrier allocation part,multiple DQN are exploited to reduced complexity of the action space.while DDPG is introduced to tackle the continuous base station(BS)beamforming matrix and IRS passive phase shift matrix.Simulation results demonstrate that the IRS can significantly improve the transmission performance of the OFDM communication system.Moreover,compared with the benchmark schemes,our proposed algorithm based on MDQN-DDPG can achieve better system transmission performance.In order to further improve the number of users available for communication service and enhance the SE over the finite spectrum resource,the spectrum sharing is introduced into IRS-assisted OFDM communication systems.The secondary users sum rate is maximized by jointly optimizing the transmit beamforming of the secondary base station,the phase shift of the IRS and the channel allocation.The introduction of spectrum sharing technology has brought the interference between primary and secondary networks,making the entire optimization problem further complicated and difficult to solve.Therefore,an algorithm based on Dueling Double Deep Q Network(D3QN)and Twin Delayed Deep Deterministic Policy Gradient(TD3)is proposed to speed up training speed and effectively improve the stability of the algorithm convergence.Simulation results demonstrate that our proposed algorithm based on D3QN-TD3 can achieve higher secondary user sum rate compared with the benchmark schemes,and the algorithm convergence effect is good.Due to the openness of wireless spectrum,wireless communication is more vulnerable to malicious attacks,therefore,the physical layer security(PLS)problem has become increasingly prominent.This thesis studies the physical layer security design of IRS-assisted OFDM system,in which the channel state of the eavesdropping user is unknown.A friendly jammer is introduced to suppress the eavesdropping user.The legitimate users secure rate is maximized by jointly optimizing the transmit beamforming of the base station,the beamforming matrix of the friendly jammer,the phase shift of the IRS and the channel allocation.Therefore,an algorithm of MD3QN-MASC based on distributed framework is proposed.The friendly jammer,IRS and base station are divided into independent units,while multiple Actor networks are introduced for distributed processing to avoid unnecessary correlation due to the same network parameters.A many-to-one architecture of multiple Actor networks corresponding to a single Critic network is proposed,while a single Critic network is employed for evaluation action to speed up the training speed of the algorithm.The distributed D3 QN network is applied to tackle the problem of the discrete channel allocation.Simulation results demonstrate that our proposed algorithm can significantly improve the secure transmission rate of the legitimate user,and achieve better security transmission performance compared with the benchmark schemes.