Research On Modeling Complex Network Evolution Mechanism Based On Reinforcement Learning

There are many complex systems in nature and human society,most of which can be abstractly modeled as complex networks composed of nodes and connections between them.Common examples of complex networks include social networks,internet networks,city traffic networks,and gene regulation networks.The rise of complex network theory provides a new perspective for the study of complex system science,and has significant implications for understanding natural phenomena and social operating rules.Related research has been widely applied in fields such as social computing,brain network research,intelligent transportation,and bioinformatics.As an abstract description of complex systems,complex networks emphasize the topological features of the system structure.Modeling the dynamic evolution mechanism of complex network topology is the basis for understanding network structure and controlling network function,as well as a frontier hot and difficult issue in the study of complex networks.On the one hand,network models help to reveal the internal mechanisms driving the dynamic evolution of network topology.On the other hand,since network structure supports physical behavior on the network,network models also play an important role in studying network behavior.Therefore,an accurate network model is the basis for understanding network structure and function,but existing complex network evolution models still have some limitations.This thesis introduces reinforcement learning and deep reinforcement learning to construct complex network evolution models and demonstrates the effectiveness and rationality of the proposed model through a large number of experiments.The main research contents and innovative points of this thesis are as follows:1.Current network evolution models are mainly constructed by adding mathematical rules based on the observation of real network characteristics by network scientists.However,these models have limitations in that they ignore the fact that nodes in the network can make independent decisions as intelligent individuals,and cannot reveal the underlying mechanisms of the emergence and evolution of communities in real networks.To address these limitations,this thesis proposes a Reinforcement Learning-based Complex Network Evolution(RDQL)model.Specifically,nodes in the complex network are modeled as intelligent agents in reinforcement learning,interacting with the environment to make decisions that drive network evolution in a low-dimensional space.A mechanism for converting low-dimensional vectors to high-dimensional network topology structures is also proposed.Experimental results show that the RDQL model reproduces the emergence and evolution of community structures in the network,and the generated network has characteristics such as scale-free and small-world properties.Additionally,a detailed analysis of the model’s parameters is presented in this thesis.2.While Q-learning algorithm is useful for constructing network models,it still has limitations,such as the need for manual feature design,dimensionality issues with high-dimensional features,limited processing ability as state space increases,inability to handle complex decision tasks,and poor generalization performance for new states or tasks.To address these limitations and consider the limitations of reinforcement learning,we propose a Complex Network Evolution model based on deep reinforcement learning(NMDRL).The introduction of deep reinforcement learning allows network nodes to have basic decision-making abilities and automatic feature learning,and to handle high-dimensional state spaces,complex decision-making tasks,and better generalization for unfamiliar states.This makes the agent’s decision-making more intelligent.Experimental results show that the NMDRL model,with the advantages of deep reinforcement learning,can generate networks with real-world characteristics such as scale-free,small-world,and community evolution,while effectively modeling complex network evolution mechanisms.This research demonstrates the validity of introducing deep reinforcement learning for complex network modeling.Compared with the RDQL model,the NMDRL model can handle higher dimensional state space,has stronger environment awareness and better generalization ability,and can be unified under the NMDRL model framework after theoretical and evolutionary analysis.3.The network models constructed based on Q-learning or DQN only make decisions based on their own environmental state,ignoring the information of surrounding agents.This means that the behavior strategy of the node is isolated,while in the real world,the behavior strategies of the nodes are interdependent,involving various cooperative and competitive relationships.Multi-agent deep reinforcement learning,combined with cooperative control in the field of multi-agent systems,is a powerful tool for modeling collective behavior,which can solve both autonomous decision-making and strategy coordination among nodes.Nodes with autonomous decision-making capabilities interact cooperatively,which is the fundamental driving force for network evolution.To address the limitations mentioned above and combine the advantages of multi-agent deep reinforcement learning,we propose a communication-based fully distributed deep reinforcement learning model for complex network evolution(ComaNetDRL+).Furthermore,we address the issue of discrete space modeling by introducing partially observable continuous low-dimensional environmental space.Experimental results show that the ComaNetDRL+model can generate networks with small-world characteristics and realistic network evolution features,as well as exhibiting emergent commumity structure.By comparing with the RDQL model and the NMDRL model,it is shown that the ComaNetDRL+ model has better collaborative decision-making capability,stronger ability to adapt to complex environments,and more accurate evolutionary mechanisms.