Research On The Applications Of Machine Learning In Network Intrusion Detection

With the continuous development of the Internet,network technology is also developing rapidly,people can use many network applications.At the same time,threats on the Internet are also emerging.Traditional intrusion detection schemes based on predefined rules cannot meet people's security requirements.As a new type of security tool,intrusion detection system has received more and more attention.Although the intrusion detection system has had a lot of research at the algorithm level and the data level,there are still some problems at present,such as the lack of research on the case of extreme imbalance of normal anomaly data and the big data environment,and intrusion detection dataset in the problem of abnormal traffic classification of the set,there is a lack of research on what types of feature combinations are effective or redundant.To solve the problem of anomaly detection in big data environment,this thesis proposes an ensemble learning Stacking OD algorithm,which is enhanced by outlier detection.Isolation forest using unsupervised learning can quickly detect outliers from the dataset,thereby enhancing the effectiveness of the model in detecting abnormal traffic.The Stacking OD algorithm achieved 99.53% accuracy,97.46% detection rate,and 0.21% false alarm rate on the UNSW-NB15 dataset,and achieved 99.73% accuracy and 99.67% detection rate and a false alarm rate of 0.23% on the NSL-KDD dataset.Compared with the same type of research,it has better performance.In order to facilitate the study of features,this thesis uses the Breaking Point tool to simulate attack traffic to generate a new data set.By verifying a variety of machine learning algorithms,it is shown that the multiclass Light GBM model can achieve the best results.In the study of feature selection,by constructing more features and conducting comparative experiments,it shows that the features provided by the UNSW-NB15 dataset are still insufficient to fully describe the attack traffic,and by caculating more features,it can further improve the classification performance.In the study of feature importance in the classification of abnormal traffic,this thesis illustrates some of the problems in the traditional feature selection scheme.by applying a variety of feature selection algorithms.If a large number of features are removed,the model performance of the model will be significantly reduced,and if the number of selected features is small,the combination and ranking of the features obtained by different feature selection algorithms are diverse,but these algorithms can reach agreement on completely useless redundant features,and experiments have verified that removing totally useless features can reduce losses of the prediction accuracy of the model.