Research On Key Technology Of Network Threat Detection And Trend Prediction

The network threats undermine the security of the network system environmentgoals or event. The threat is a potential attack, often both equivalent, but in this article,threats including attacks intention connotation. The frequent cyber threats give a greatrising harm to Internet security, which has led Cyber Threat Detection to become a hottopic recently. Cyber threat detection, firstly gathers potential cyber threat activityinformation in various ways, then uses multiple analyzing, grouping and detectingtechniques to identify the cyber threats based on the threat signatures in the gatheredinformation, and finally determines the threat classification, threat level and theorigination and destination of threat. In recent, there are a lot of efforts have been giveninto this field. However, there are still some problems to be solved in the aspects ofsynergistic association analysis and system architecture, regarding to cyber threatinformation gathering and grouping, signature extraction and merge, threat behaviordetermination and detection.Addressing the problems and requirements in cyber threat detection, thisdissertation firstly reviews the state of the art of cyber threat detection techniques, andthen gives efforts on threat sensor parallel deployment algorithm, alert informationassociation technique, threat classification model and its threat behavior detectingalgorithm, synergistic detecting model and its architecture, threat prediction model anda security situation analyzing algorithm based on it. The major contributions of thisdissertation can be summarized as follows:1. A Sensor Parallel Deployment Algorithm (SPDA) for cyber threat informationgathering is proposed, and a SensorPool prototype system is implemented. Theproposed algorithm can achieve a fast parallel deployment of threat sensor. Using theSensorPool to deploy sensors is propitious to fast and effectively utilize networkresource to carry out security defense, and to improve the flexibility and effectiveness ofdeployment. Compared with the most popular Virtual Honeynets and Potemkindeployment algorithms, the time cost of the proposed SPDA is reduced dramatically,and the more deploying nodes, the more parallelism can be achieved, so the less timecost. Meanwhile, an association algorithm for alert frequency pattern mining andautomatic alert time dividing is proposed, which further processes the frequency pattern,and filters the false alarms. The time cost is reduced to1/60of the original algorithmwhile keeping more than95%accuracy of the original algorithm.2. A cyber threat detecting architecture based on threat classification and behaviorsequence template is proposed, which includes alter information process module, threatdetection module, synergistic detection module, threat prediction module and networksituation analysis module. By analyzing the Snort rule database, TIAA (a Toolkit for Intrusion Alert Analysis) system, and the feature of current cyber threat behavior, athreat classification framework is proposed, which includes initial classificationmatching model, structural semantics model, feature match reuse model, feature matchadaptive iteration model, threshold determining model, match classification model andthreat matching matrix model, etc. Specially, a detailed introduction is given to theconstruction method of the two most important modules, i.e., the initial classificationmatching model and the threat matching matrix model. Based on the threatclassification model, a construction method of threat behavior sequence template isproposed, which converts the threats in the rule database into sequences instead ofelements, providing a more flexible rule database to complex attack detection. Based onthe threat classification model and sequence template, two threat detection matchingalgorithms are proposed: pattern matching algorithm and graph matching algorithm, inorder to merge threat features and cyber threat behavior. Experimental results indicatethat the time cost of these two algorithms reduces above50%compared with the classicCupid and S-Match algorithms. The average time cost of graph matching algorithm isabout65%of pattern matching algorithm. The more graph nodes in the template and themore complex of threat detection, the smaller is the ratio of time cost of graph matchingalgorithm relative to the time cost of pattern matching algorithm.3. We define the threat synergistic detection model and constructs a ThreatDetection Layer Cooperation-TDLC model through model framework, modeling andsynergistic mechanisms. The TDLC model is introduced in details in four layers: modelframework, modeling process, data structure and synergistic mechanism. Base on themodel, a cyber-threat synergistic detecting system and its architecture is proposed, andthe design objective, architecture, logic structure, physical structure and work principleare explained in detail. Considering the current mainstream threats, i.e., botnet andDDoS attach, a distributed detection method based on synergistic model is proposed.The synergistic detection on botnet attack is built upon a synergistic sensing model. Thecooperation mechanism of threat sensors for threat detection is explained afterwards.Addressing the creditability problem that the deployed threat sensor nodes might betaken over by attackers, the malicious sensor that determining method based on trustmeasurement is proposed; addressing the synergistic detection on DDoS attack, a trafficstatus snapshot prediction algorithm, a fine-grain exception detection algorithm and amalicious IP address extraction algorithm is proposed. Experimental results indicate that,compared with the recently proposed traffic exception detection algorithm based oncomentropy and subspace method, the proposed detection algorithm can effectivelyhandle the DDoS attacks based on botnet. The detection algorithm proposed in thisdissertation has a relative high precision at the initial stage of threats; however, thedetection precision approaches the same value for both algorithms as the ratio of DDoStraffic in the background traffic increases. 4. We propose a threat prediction recognition model as the threat predictionframework. By improving the particle swarm optimization algorithm, an overlapprediction algorithm based on prediction model is proposed, and a threat predictionmodel is constructed to predict the threat trends. The prediction error by predictionmodel is about a half of the error by the particle swarm optimization model, whichsufficiently proves the accuracy and robustness of the prediction model. According tothe quantitative evaluation on the network security situation, the network can beclassified as system level, host level, service level and attack level.(We) define thethreat index for the existing threats, and propose a quantitative computation method forthreat index.(We) further calculate the importance weight for every level, and use it toevaluate the security situation for the entire network. We introduce a D-S evidencereasoning method to grade the possibilities of all the threats occurred in the network. Byidentifying the specific cyber threat class,(sth.) determines the weight ratios of all thethreat occurred in one day, and then makes a detailed analysis on the network securitysituation.This dissertation serves as an instructive practice and exploration on the cyberthreat detection techniques. The results have a theoretical and practical value onpromoting the cyber threat detection research, and it is an affirmative promotion on theperfection and development of network security.