Research On Network Congestion Control Protocol Based On Deep Reinforcement Learning

With the continuous increase in the scale and network applications of the mobile Internet and the Internet of Things,network congestion is becoming increasingly significant.As a result,network congestion has become an important issue that restricts network development.The network system itself has the characteristics of complexity,time-varying and uncertainty,and it is impractical to improve it at the physical level.In the face of network changes,the traditional TCP congestion control protocol can only make fixed actions due to its inherent rule mechanism,which neither fully utilizes link bandwidth nor historical network data,and restores bandwidth when congestion occurs.It takes a long time.Therefore,studying the problem of network congestion has far-reaching significance for improving network performance and quality.This paper mainly studies TCP New Reno protocol and how to introduce deep reinforcement learning into network congestion control.The main work of this paper consists of three parts:(1)This paper presents a congestion control scheme based on the deep reinforcement learning method.Based on this method,the size of congestion window is controlled to control the rate of data transmission.At the same time,based on the delay in the learning process,this paper modifies the Q function of iterative update to better explain the model.(2)Based on the historical data in the network,there is a potential impact on the future network planning.In this paper,the implicit causality of historical data in a certain period of time is extracted based on temporal convolution network,and then the intelligent congestion control is realized by combining reinforcement learning method.When selecting a certain amount of data,we can find the effect of historical data better.(3)This paper proposes a reinforcement learning congestion control scheme based on graph structure.When the graph attention network deals with the network topology,the importance of the link to the network must also be considered.Therefore,this paper adds a side attention network module based on the graph attention network structure.The main purpose of graph attention network is to extract the features of intermediate nodes and links.These features and network parameters can be used as input for reinforcement learning methods.Experiments show that the graph attention network has a certain generalization ability when dealing with changing network topologies.In this paper,a large number of experiments prove the feasibility and effectiveness of this scheme,and compared with the existing model methods,it verifies that the proposed model has a significant improvement in throughput and transmission delay.In the face of complex and changeable network,the proposed scheme can also get competitive results,thus breaking the limitations of traditional network congestion control.