Research And Construction On Network Security Big Data Target System

With the rapid development of computer network technology,the network brings convenience and convenience to people's lives and work.However,security issues cannot be ignored.The vulnerability information of various hardware and software systems is the most important content in network security.Most hacking activities are initiated based on the vulnerabilities of specific information systems.Therefore,the timely acquisition and effective use of network security vulnerability information is an important method for network system protection.Various countries and social organizations have invested a large amount of resources to build a network security vulnerability database,using network attacking and defense platforms to train network security personnel,and the target environment plays an important role in practice as a core component of attacking and defense platforms.At present,there are many vulnerabilities data publishing platforms at home and abroad,and there is a large volume of vulnerabilities,but they do not follow a unified standard.There are a large number of heterogeneous and redundant vulnerabilities.The publicly released vulnerability targets on the Internet are less frequent,and the update frequency is low.The target environment in attacking and defense platforms is mostly based on the CTF problem-solving model.The target environment is modeled and biased towards academics.Targeted environments with real vulnerability are even less.In addition,the major challenge facing the construction of a real vulnerability target environment is the dependence on historical environment software configuration.There is no mature technology model in this field that can systematically solve the problems we face.This paper conducts in-depth research on the vulnerability data collection technology,data standardization cleaning technology,vulnerability data fusion technology,vulnerability database construction technology,and target environment construction technology,etc.,and analyzes the basic vulnerability database security knowledge system and security content automation protocol at home and abroad.On the above,we designed and implemented the construction of a network security large data set and the construction of a target system based on this large data set.The specific research work of this article is as follows:1.Designed and implemented a standardized vulnerability data collection module,extensively studied vulnerability data characteristics of major vulnerability publishing platforms at home and abroad,and used web crawler technology to realize large-scale real-time and stable collection of original vulnerability data of each vulnerability database.Based on the characteristics of vulnerability data,the data cleaning module is used to implement standardized data processing for vulnerability data,and provides a stable and standard vulnerability data source for building a network security large data set and target system with a security knowledge system.At the same time,according to the system design requirements,the automated processing technology of vulnerability data and database storage technology are implemented to implement the corresponding functional modules.2.Build a network security large data set with a security knowledge system,analyze the construction techniques and vulnerability data characteristics of major vulnerability databases at home and abroad from multiple levels,and propose a vulnerability information correlation technology based on a combination of a common vulnerability number and a specific vulnerability number.And comprehensively analyze each vulnerability database to evaluate the comprehensiveness and coverage of the vulnerability data covered by the vulnerability.Based on the assessment results,a network security large data set with a security knowledge base including fields of vulnerability data sources,data references,SCAP support,POC conditions,platform environment affected by vulnerabilities,and exploit code conditions was established.3.Designed and implemented a vulnerability target environment building module.The system is based on a network security large data set with a security knowledge system built in the first two steps to study the conditions and technologies for the construction of the target environment,and analyzed and complemented the target environment.The essential elements needed to extract the vulnerability data that can be used to construct the target environment.Based on this,the research and use of Docker host virtualization technology,software dependent time backtracking technology in the Linux system environment to solve a series of problems in the target environment construction,and finally a vulnerability target environment building module was implemented.Based on the above three parts of the research content,this paper implements research and construction based on network security big data target system.The system's vulnerability data collection module,cleaning module,vulnerability database building module,and vulnerability target environment building module work closely together to achieve timely and comprehensive awareness of network security vulnerability information.Standardization and modularization create a target environment with real vulnerability.Provides strong data support and environmental support for the research of vulnerability information in the network security field and the actual training of network security personnel.