Research On Anomaly Traffic Detection Technology Based On Mahout Classifier

In recent years,the Internet brings abundant information resources to people’s life and it makes people’s work more convenience and comfortable,a large numble of people cannot do without the network.Network is playing a more and more important role in modern people’s life.On the other hand,a lot of people against the network by using a bad way to attack the network for getting the benefits.For most ordinary users who are in the increasingly complex network environment,not only need to strengthen their own protection consciousness,but also need more network security workers to support network maintenance and supervision,so as to ensure the user’s network security.Therefore,the network anomaly traffic detection technology is of great significance,it is also the main research topic of the text.According to the multidimensional characteristics of the network data,we build the multi-dimensional information entropy projection of multidimensional data source data to different classification support vector machine learning.According to itself characters of "under learning" and "over learning",we compared with EWMA,Entropy,K-means,GMM and SVDD anomaly detection methods,selected a good method to handle high dimensional data,it is an unsupervised SVDD classification method of strong generalization ability.In view of the inherent conflict between the high cost and low efficiency of the classifier,we choose a kind learning pattern of abnormal response training,only when there are abnormal detection points to be re-trained.Not only that,for the selection of outliers in training data,Bayesian network model is used to predict the probability of the next node,and then judge whether the abnormal points will be added to the training center to continue training,so as to improve the accuracy of abnormal flow detection.In this paper,the main problems and innovative points are as follows:(1)According to the classification based traffic anomaly detection method,the training data is difficult to obtain,and the data analysis process is complicated.We choose large data distributed platform environment for data processing and analysis.According to the detection results in anomaly detection classifier,the anomaly detection method compared with EWMA,Entropy,K-means,GMM,SVDD and other mainstream methods,the final experiment built support vector based on multidimensional information entropy,using SVDD method to detect the abnormal traffic,the result can conform that the method can effectively improve the detection accuracy and outperforms than other methods.(2)According to these problems: with the change of time space and deployment of the classifier,the original training set can not adapt to the new data,this paper puts forward the training method of abnormal response.Only when the abnormal points are added,the training set is re-trained,and the latest detection vector is added,the oldest detection vector would be deleted.This method can not only improve the adaptability and accuracy of the training set,but also reduce the material consumption and resource consumption.(3)According to the problem of detecting abnormal sample of statistic time correlation and accuracy,this paper uses the Bayesian network model to predict abnormal nodes,to make the detection accuracy optimized,then re-add abnormal node into training set.According to the anomaly detected by anomaly detection model,this paper puts forward the method of emergency response,which can not only detect anomalies,but also can deal with the anomalies.