Research On Network Deception Defense Method Based On Game Model And Deep Reinforcement Learning

The network information system has become a critical national infrastructure,while network security issues are constantly emerging,with network security threats becoming increasingly severe.Due to the lagging and static nature of traditional network security defense methods,the cyberspace is facing a security situation of "easy to attack and difficult to defend".Network deception defense,without altering the existing network framework,deploys deception resources that simulate real network systems but lack real business to lure attackers,providing a new approach for proactive security defense of information systems.However,the current network deception defense lacks effective and accurate modeling methods in terms of strategy selection,strategy implementation,and performance evaluation,and existing solution methods lack capabilities of feedback learning and evolution,failing to meet the demands for automation and intelligence in decision-making for large-scale network attack-defense confrontations.Therefore,there is an urgent need to design and develop models and algorithms for modeling and solving network deception defense.Game theory is a mathematical theory that addresses strategy optimization problems under specific constraints,providing a scientific model for studying network attack-defense confrontations.Deep reinforcement learning algorithms possess the capability for strategy learning evolution and intelligent problem-solving,which can enhance the efficiency and performance of network deception defense methods.This study focuses on researching network deception defense methods based on game theory and deep reinforcement learning technology,working across three dimensions: identifying scientific problems,constructing game models,and advancing solution algorithms.The research is centered on the selection of deception timing,deception resource selection,implementation of deception defense,and evaluation of deception effects.Network attack-defense modeling is conducted based on Flip It games,differential games,delayed differential games,and Stackelberg games.The study utilizes Proximal Policy Optimization(PPO),Deep Q-Network(DQN),Neural Fictitious Self Play(NFSP),and Deep Deterministic Policy Gradient(DDPG)for the selection,implementation,and evaluation of deception strategies.The research answers four key questions regarding network deception defense: when to deceive,what to deceive with,how to implement deception,and the effectiveness of deception.The primary work includes the following aspects:1.To address the issue of existing deception timing selection methods being unable to adaptively adjust to strategy changes in the attack phase,a deception timing selection method based on Flip It games and PPO is proposed.Firstly,based on the analysis of network deception attack-defense behaviors,a network deception model based on moving deception attack surfaces is presented,and the interaction behaviors of network deception attack-defense are analyzed.Subsequently,a single-stage network deception timing decision model is established using Flip It games,and a multi-stage network deception game model is constructed by introducing discount factors and transition probabilities.Furthermore,a payoff function for the multi-stage network deception model is developed,and an algorithm for solving the optimal deception defense timing strategy is designed based on PPO.Finally,through application examples,it is verified that the proposed model and method can provide optimal deception timing strategies for 8 different attack stages,with a computational latency reduction of over 25% compared to other algorithms.2.To address the problem of difficult and inefficient real-time selection of deception resources in continuous dynamic network attack and defense scenarios,a deception resource selection method based on differential game and DQN is proposed.Firstly,an improved infectious disease model is employed to analyze the evolutionary process of network security states and establish differential equations for the states of nodes within the network system.Secondly,a differential game model for network deception attack and defense is constructed,followed by the design of an attack-defense payoff function.Subsequently,an optimal solution for the strategy of deception defense resource allocation is designed based on DQN algorithm.In conclusion,through instances of microservice offense and defense within a cloud-native environment,the proposed model and methodology have been validated to provide optimal deception defense strategies in real-time,tailored to the network environment’s state at varying times.Compared to other deep reinforcement learning algorithms,the convergence speed of the proposed method has been enhanced by more than 77.8%.3.In response to the limitations of current deception defense implementation methods,which fail to adequately model complex network scenarios with inherent time delays and struggle to adaptively adjust strategies,a deception defense implementation approach utilizing time-delay differential game and NFSP is proposed.The state evolution process of network nodes within complex network environments is initially analyzed,leading to the derivation of time-delay differential state equations for nodes with varying degrees.A model for time-delay differential game in network deception attack-defense scenarios is then constructed,incorporating specifically designed payoff functions for both the attacking and defending entities.Through the application of the NFSP algorithm,an optimal strategy for network deception implementation is developed.Experimental results have demonstrated that the method proposed in this paper is capable of accurately characterizing the evolutionary process of strategies employed by both attackers and defenders,as well as the changing state of network security,in network structures with nodes of varying degrees.In comparison to other deep reinforcement learning algorithms,the method introduced herein has been shown to enhance network defense gains by no less than 82%.4.In response to the challenge where current methods for evaluating the effectiveness of deception fail to combine model analysis with attack-defense experimental analysis,a evaluation approach employing Stackelberg games and Deep Deterministic Policy Gradient(DDPG)is introduced.Initially,an analysis of the adversarial model between attackers and defenders leads to the development of a network deception defense effectiveness evaluation model based on Stackelberg games,with the payoff functions for both parties being designed in accordance with the Common Vulnerability Scoring System.Following this,by integrating performance and security impact factors,a multi-objective optimization function is established,and a method for evaluating optimal network deception defense strategies is devised using the DDPG algorithm.Moreover,a simulation experiment environment for confrontation,utilizing the Network Attack Simulator(NASim),is designed and executed,providing an in-depth overview of its architecture and procedural flow.In conclusion,the efficacy of the model and methodologies proposed in this paper was verified.Defender strategies were implemented within a simulated experimental setting,facilitating the assessment of defense success rates and defensive gains under a variety of influencing factors.Compared to pre-existing methods,it was observed that the defense success rate was enhanced by over 16%.