Research On Intelligent Routing Method Based On Deep Reinforcement Learning For SDN

With the rapid development of the Internet,new network applications and service requirements continue to diversify,network load and energy consumption have increased sharply,and traditional routing algorithms have become difficult to guarantee service quality.Software-defined networking(SDN)is a new type of network architecture that can obtain a global view of the network to achieve more fine-grained network monitoring and flexible deployment of network functions.With the rise of artificial intelligence,the SDN intelligent routing algorithm based on machine learning has gradually shown its development potential.However,it still needs to be improved in terms of performance optimization and promotion of network transmission delay,throughput and link utilization,and reduction of energy consumption.Therefore,this paper proposes an intelligent routing method based on deep reinforcement learning in the SDN environment.The main research work is as follows:(1)Aiming at the problems of slow convergence speed,high average delay and low throughput of existing routing algorithms,this paper proposes an intelligent routing algorithm RDPG-Route for load balancing.In order to effectively reduce the training time and enhance the training effect at the same time,RDPG-Route adopts recurrent deterministic policy gradient(RDPG)as the training framework and introduces long short-term memory(LSTM)as the neural network.Experiments show that RDPG-Route has better convergence and effectiveness.Compared with other better routing algorithms,it reduces the average end-to-end delay by 7.2%and increases throughput by 6.5%.(2)In order to further improve the generalization ability of the load balancing routing algorithm,this paper proposes an application-oriented generalized intelligent routing algorithm GNN-DRL.For the generalization of training experience,GNN-DRL adopts deep deterministic policy gradient(DDPG)as the training framework,and introduces Graph Neural Network(GNN)to perceive the dynamically changing network topology.Experiments show that GNNDRL reduces the maximum link utilization by 13.92% and the average end-to-end delay by9.48% compared with other optimal routing algorithms,and can extend the training experience to different network topologies.(3)In order to further enhance the energy-saving advantage of the load balancing routing algorithm,this paper proposes an energy-saving intelligent routing algorithm Ee-Routing.EeRouting uses DDPG as the training framework.In order to improve the convergence efficiency of the routing algorithm,a convolutional neural network(CNN)is introduced for training.EeRouting regards network energy consumption and performance as joint optimization goals,and establishes energy-efficient traffic scheduling schemes for elephant-flows and mice-flows respectively.Experiments show that compared with other optimal routing algorithms,EeRouting increases energy saving percentage by 13.93%,reduces delay by 13.73%,increases throughput by 10.91%,and reduces packet loss rate by 13.51%.