Non-Invasive Strategic Congestion Control Research Based On Beep Reinforcement Learning

With the development of emerging information technologies such as cloud computing,big data,and the Internet of Things,massive amounts of multi-source heterogeneous data are generated in various fields,but due to the limited infrastructure resources of communication networks,data access to the network is more likely to cause congestion.The currently applied congestion control protocols cannot effectively achieve low latency,high throughput and continuous reliable control when dealing with different transmission requirements in heterogeneous network communication.Therefore,this paper introduces deep reinforcement learning algorithm into the congestion control algorithm,and proposes a congestion control algorithm that can fairly allocate network resources by effectively identifying network congestion through the dynamic sensing of network environment changes by an intelligent body and eliminating environmental noise interference in the sensing process.The main work and innovation points of this paper are as follows:(1)A differential congestion control algorithm based on deep reinforcement learning is proposed for the problem of adaptive heterogeneous networks.Then the agent senses and learns the network environment state data autonomously by deep reinforcement learning algorithm,and finally decides the optimal action according to the cumulative reward effect,and then achieves the most favorable congestion control for the network.The experimental results show that this method has better results in fast sensing network environment and bandwidth utilization compared with traditional congestion control algorithms.(2)The temporal feature extraction module is designed to address the problem of difficulty in extracting feature information from dynamic and complex networks.In order to better obtain implicit temporal continuity feature information from the network history data,this study takes the traffic prediction results as part of the network state information,designs an action network with temporal sequence feature analysis capability based on convolutional neural networks and gated recurrent units,and designs an evaluation network that takes into account the state value and state action advantage value,which makes the algorithm of this study more reasonable in the policy inference stage and The learning efficiency of the algorithm is improved.The experimental results show that this algorithm converges quickly and outperforms the traditional congestion control algorithm.(3)The fair congestion control algorithm based on a multi-intelligent body system is proposed for the fairness problem that arises when multiple data streams compete for link scenarios.Firstly,the multi-stream competitive link scenario is modeled as a Markov game process,and the multi-agent reinforcement learning algorithm is used to learn the optimal control strategy based on the local state and global joint state information.Then the idea of double delayed policy gradient is used to introduce gradually decaying Gaussian noise to go for state action value smoothing between actions,avoiding the misestimation of state value function.Finally,experiments show that this research algorithm has good fairness when multiple data streams compete for link bandwidth and can achieve reasonable allocation of bandwidth.In summary,the method proposed in this paper has the ability to adapt to network changes in a heterogeneous network communication while maintaining high throughput,low latency and low delay,and can allocate network resources in a balanced manner in multi-stream competition for link bandwidth.