| Ad-Hoc network,also known as mobile ad hoc network,is a dynamic network composed of mobile nodes without fixed infrastructure or routers.Nodes communicate by establishing direct connections to realize data transmission and information exchange.However,efficient routing becomes an important challenge since the connectivity and link quality between nodes varies at any time.Software-defined networking(SDN)is a network architecture and technology that realizes intelligent routing and traffic control through centralized management and control,improving network performance and reliability.Artificial intelligence-based deep reinforcement learning can be used to optimize network controllers in SDN,learn optimal decision-making strategies from historical network data and feedback information,improve the intelligent decision-making level of network controllers,and refine the granularity of network resource allocation.However,with the dynamics of Ad-Hoc networks and the substantial increase in the size of network nodes,traditional routing optimization algorithms fall into performance bottlenecks.At the same time,due to network data sparsity,state space complexity,and environmental There are also challenges.Aiming at these problems,this thesis proposes two routing control mechanisms to solve the routing problems of small-scale highly dynamic Ad-Hoc networks and large-scale cluster Ad-Hoc networks.First,this thesis proposes a deep reinforcement learning routing algorithm for graph neural networks based on message passing for small highly dynamic Ad-Hoc networks.This centralized QoS optimization routing mechanism realizes the "real-time migration fitting" of the graph neural network for the relationship between dynamic network topology and resource distribution.Secondly,this thesis proposes a "flat" distributed routing traffic optimization management and control architecture for large-scale cluster Ad-Hoc networks.Using the independently trained multi-agent deep reinforcement learning algorithm,the distributed controller independent consensus cooperative routing traffic optimization is realized.The main contributions of this thesis include:First of all,for small-scale and highly dynamic Ad-Hoc network scenarios,this thesis proposes a graph neural network deep reinforcement learning routing algorithm based on message passing,which solves the"overfitting" phenomenon of network topology on deep reinforcement learning routing algorithms,and realizes The "real-time migration and fitting" of the relationship between dynamic network topology and resource distribution is realized,and the problem of optimization convergence difficulties brought by the uncertainty of the interactive environment to deep learning algorithms is solved.Secondly,for large-scale cluster Ad-Hoc networks,this thesis proposes a distributed multi-agent routing optimization algorithm based on IPPO(Independent Proximal Policy Optimization),and uses distributed training and distributed execution of the multi-agent algorithm to perform partitioned and sub-regional routing of Ad-Hoc network routing Collaborative optimization realizes distributed software-defined network controller independent consensus collaborative routing traffic optimization.Finally,this thesis verifies the effectiveness of the proposed routing optimization algorithm through simulation.The experiment adopts the simulation environment and reinforcement learning training environment developed by Python,and compares with the existing scheme on relevant indicators.After simulation experiments,compared with traditional path selection algorithms and other deep reinforcement learning algorithms,the algorithm proposed in this thesis has advantages in many indicators such as convergent network throughput and end-to-end delay. |
