Reinforcement Learning Based Inter-cell Interference Mitigation Technology Study On Ultra-dense Heterogeneous Networks

With the trend of dense network deployment,inter-cell interference becomes a key factor to limit the communication performance of ultra-dense heterogeneous networks.Conventional interference mitigation technologies are difficult to optimize the long-term communication performance of networks dynamically based on the knowledge of the inter-interference distribution and the assumption of the static channel states and the uniform flow rate.In addition,a practical interference mitigation scheme should reduce the computational resource and the communication overhead of base stations(BSs)with many wireless devices.Therefore,this paper studies the inter-cell interference mitigation technology to improve the communication reliability and the energy efficiency for ultradense heterogeneous networks.First,a reinforcement learning based interference mitigation scheme is proposed,in which each small cell BS makes full use of the observed network state and the interference mitigation experience to solve the performance bottleneck caused by the inaccurate interference characteristics and the assumption of the static network.More specifically,the small cell BS applies reinforcement learning to optimize the interference mitigation strategies based on the receiving signal-to-interference-plus-noise ratio of local users,the local channel power gains,and the user density.This scheme combines transfer learning to reuse the interference mitigation experiences for improving the small cell throughput with less energy consumption in the dynamic networks.This paper provides the optimal inter-cell interference mitigation strategies with conditions via theoretical analysis.The effect of the channel power gain,the unit transmission cost and the system bandwidth on the resulting performance bound is also analyzed.Simulation results based on the small cell density 1300 cells/km2 show that the proposed reinforcement learning based interference mitigation scheme can improve the small cell throughput by about 17.3%and reduce the energy consumption by about 11.5%compared with the existing Bi-SON scheme.Furthermore,a deep reinforcement learning based interference mitigation scheme is proposed for small cell BSs that can support the computational complexity of deep learning.The actor-critic network is applied to obtain the high-dimensional state features and decouple the strategy selection and evaluation without introducing the quantization errors of channel power gains and falling into local optimal solutions.This scheme can accelerate the learning speed of the BS and improve the network capacity.Simulation results based on the small cell density 1300 cells/km2 show that the proposed deep reinforcement learning-based interference mitigation scheme further reduces the intercell interference by about 23.5%and improves the small cell throughput by about 16.7%compared with the proposed reinforcement learning based scheme.