Research Of Semi-Supervised Learning For Reducing False Positives

Nowadays, with the popularization of internet and development of computertechnology, a variety of information security and network security risks are also becomingmore prominent. As one of important components of the network security system, intrusiondetection system can ensure the security of computers by analyzing security of the networkdata and detecting attack behavior. However, intrusion detection system generates a largenumber of false positive alerts which reduce its efficiency. How to reduce the high falsealarm rate produced by the intrusion detection system is a problem researchers widelyconcern. There exist the following three limitations in the most of the methods available.Firstly, they need a lot of class labeled training data or domain knowledge to build thealarm filtering model. However, it is difficult to obtain sufficient labeled training data.Secondly, because most of them are offline model, the response to processing of aggressivebehaviors is delayed. Lastly, intrusion patterns constantly change, which makes the modeldifficult to find a lot of intrusion attacks. The above deficiencies lead to the high falsepositive rate of the existing intrusion detection systems.In order to reduce the high false alarm rate generated by IDS, this paper designs amethod of alarm filtering based on semi-supervised learning. First of all, we use the limitednumber of labeled alert data to estimate the parameter of alert generative model and buildNaive Bayesian alert classifying model. Secondly, this paper makes use of alert classifyingmodel built to train and sign the non-labeled alert data,obtain new labeled alert data. Finally,we use all of labeled alert datas to estimate the parameter of alert generative model again,then update the alert classification model. The algorithm iterates the above three steps untilthe members in the labeled dataset do not change much from one iteration to the next. Theexperimental results demonstrate that our method can achieve better alert classificationperformance using only a few labeled alerts combined with sufficient unlabeled alerts,reduce the false alarm rate generated by IDS.The high dimensional complexity of raw alert data impact on efficiency andperformance of the model, and lead to the dimension disaster easilly.This paper designs anew dimensionality reduction clustering algorithm based on semi-supervised learning.Firstly, semi-supervised dimensionality reduction framework is used to project original datasamples into lower dimensional space, and then semi-supervised clustering is performed inthe reduced space. The proposed semi-supervised dimensionality reduction is composed ofa discrimination term that evaluates the separability between the classes and a regularization term that characterizes some property of the original data samples. Owing toboth dimensionality reduction and clustering procedures have taken advantage of thesupervision information, the proposed method can achieve further improvement onclustering performance.The proposed alert classification model is tested on the well-known KDD CUP99dataset. The original alert data dimensionality are reduced by using dimensionalityreduction algorithm based on semi-supervised, which avoids the "curse of dimensionality"problem and reduces the computational complexity. The data processed by dimensionalityreduction algorithm, are classified by the alarm classifying model based onsemi-supervised learning. The experimental results show that the performance of the alarmclassifying model proposed by this paper is superior to traditional alert classifying methods.The model can achieve better alert classification performance using only fewer labeledalert combined with sufficient unlabeled alerts, reduce false alarm rate generated by IDS.