Deep Reinforcement Learning-based Performance Optimization For IRS-assisted Wireless Communication Systems

Intelligent reflecting surface（IRS）is a revolutionary technology to improve the performance of wireless communication by adjusting the amplitude and phase of the incident signal to reconfigure the wireless channel environment.However,due to a large number of reflecting units in IRS,the design of convex optimization algorithms is often difficult and complex.In addition,the different optimization problems of different communication systems require different optimization algorithms,which involve different mathematical derivations and algorithm design processes.To reduce the algorithm’s design difficulty and computational complexity,this thesis proposes a kind of artificial intelligence algorithm called deep reinforcement learning（DRL）to solve the IRS-assisted communication system problems.We investigate the performance optimization of two IRS-assisted wireless communication systems by applying the deterministic policy-based DRL algorithm called Deep Deterministic Policy Gradient（DDPG）algorithm and the stochastic policy-based DRL algorithm called Soft Actor-Critic（SAC）algorithm to optimize the IRS reflecting phase shifts and the corresponding communication resource allocation in the two different IRS-assisted wireless communication systems to improve their communication performance.The main contents of this thesis are as follows.1)We study the IRS-assisted cognitive radio（CR）communication system by optimizing the transmit power of the primary transmitter and the IRS reflecting phase shifts to maximize the signal to interference plus noise ratio（SINR）of the secondary receiver while ensuring the quality of service（Qo S）of the primary user.This thesis first transforms the optimization problem into a reinforcement learning（RL）problem,and then solves the optimization problem by using the DDPG algorithm and the SAC algorithm,respectively.In addition,a method is proposed to improve the performance of the DRL algorithm by multiplying the reward in the Bellman equation by an appropriate factor to improve the learning efficiency and stability of the two DRL algorithms and reduce the accumulated reward variance.Simulation results show that the two proposed DRL algorithms can achieve a comparable communication rate of the secondary user without using complex mathematical derivation techniques and a large amount of computation,compared to an existing convex optimization algorithm.Moreover,the complexity of the two DRL algorithms is much lower than that of the convex optimization algorithm as the number of IRS reflecting units increases.Meanwhile,the simulation results show that the SAC algorithm outperforms the DDPG algorithm in terms of learning efficiency,stability,and reduction of the reward variance.2)To investigate whether the DRL algorithm can be used to optimize different wireless communication systems,this thesis investigates the IRS-assisted secure communication systems.By using the DRL algorithm framework in the previous work,DDPG and SAC-based algorithms are proposed to maximize the secure rate of the communication system by optimizing the transmit power of the transmitter and the IRS reflecting phase shifts,respectively,without changing the two DRL algorithms framework and adjusting the hyperparameters of the two DRL algorithms extensively.We only need to change the state,action,and the reward of the agents of the two DRL algorithms according to the system model.Simulation results show that the DDPG algorithm and SAC algorithm can achieve better secure communication rates than an existing convex optimization method with lower computational complexities.Simulation results also verify that the SAC algorithm outperforms the DDPG algorithm in terms of the learning efficiency,stability,and reduction of the reward variance.Therefore,this thesis has demonstrated that the DRL algorithm has good generalizability in optimizing different IRS-assisted wireless communication systems.