Research On Network Intrusion Detection Method Based On Feature Extraction

With the rapid development of new technologies such as big data,cloud computing and5 g,the number of network traffic has soared.Network flow records all information of network communication,including network attack behavior.It is an ideal data for intrusion detection.The network traffic in the real network environment has the characteristics of large amount of data and high dimension.The direct analysis of it will consume a lot of computing resources and reduce the efficiency.This thesis proposes a new technology to extract the prominent feature of network flow.It is combined with traditional machine learning methods and verified by experiments on public datasets.It is also combined with unsupervised anomaly detection technology and applied to the real network environment.To address the problems of high reconstruction error and long training time when using Stack Non-symmetric Deep Autoencoder(SNDAE)feature extraction technology for intrusion detection,an Adam Non-symmetric Deep Autoencoder(ANDAE)is proposed based on SNDAE.The Adam optimization algorithm is used to update network parameters during training,so that the loss function can quickly converge to the ideal value.Under the premise of not affecting the effect of feature extraction,the network structure is simplified and the training time of the network is reduced to realize the efficient extraction of the rapid growth of high-dimension and nonlinear network traffic features.For the low-dimensional prominent features extracted by ANDAE,Random Forest is used for classification to detect intrusion action,and a network intrusion detection model based on ANDAE feature extraction is implemented.The experimental results on the NSL-KDD and the CIC-IDS2017 datasets show that compared to the SNDAE-based intrusion detection model,the ANDAE model has an average increase of 6.78% in accuracy,an average of 13.06% in recall,and an average of14.9% in F1 scores.Feature extraction time is reduced by 23.1% on average.Thus the ANDAE model is an intrusion detection solution which can simultaneously improve detection accuracy and time efficiency.Aiming at the problems of long running time and low detection accuracy caused by the disaster of data dimension when using traditional anomaly detection methods to detect real network data,an unsupervised intrusion detection model(ACOPOD)based on ANDAE feature extraction and COPOD algorithm is proposed.The model uses ANDAE to extract the features of the preprocessed real network data,extract meaningful features and reduce the data dimension.The COPOD algorithm based on probability is used to detect the anomalies of the extracted data,so as to reduce the running time of the algorithm and improve the accuracy of the algorithm.Compared with other six traditional machine learning models on NSL-KDD dataset,ACOPOD has achieved high accuracy and ROC-AUC score,as well as low running time.The experimental results on the real network dataset with a time span of one month captured from the ATC intranet show that this method can effectively detect the intrusion behavior in the real network environment.