Research On Network Anomaly Detection Based On Deep Transfer Learning

Network attack is a serious problem in information society nowdays.Intrusion detection system is an important guarantee for network security.Recent studies have applied deep learning to network intrusion detection,which has great improvement compared with traditional machine learning algorithms based on rule signatures and feature engineering.However,these methods take the imbalance of anomaly samples into account rarely.Therefore,it is difficult to ensure the effectiveness of the algorithm on the training set without enough anomaly samples.At the same time,due to the feature shift in the network traffic among different scenarios,it is difficult to apply the models trained on a network flow data set in a new scenario directly.Generative adversarial network(GAN)as an unsupervised deep learning method has achieved good results in many fields.In order to solve the above problems,this paper proposes a GAN-based deep transfer learning network anomaly detection framework.Evaluated on different public data sets,the algorithm achieves satisfied performance.The main research work and contributions of this paper are listed as follows.Aiming at solving the imbalance problem on network flow data,an improved GAN is used to generate simulated anomaly samples.Besides,the convolutional neural network(CNN)and Gate Recurrent Unit(GRU)are used to capture the spatio-temporal feature.When the anomaly samples in the training set are used up,the generated simulated anomaly samples will be used to fill in.The input of model can achieve the self-balance of the sample ratio by this way.Experiments on multiple data sets show that the method proposed in this paper can achieve an average anomaly detection rate of 92.65% and detection accuracy of 90.11% on data sets with anomaly sample of less than 10%.The effect of anomaly sample ratio in training data set for the performance of algorithm is also analyzed in this paper.In order to solve the problem of mode shift caused by the dynamic changes of data characteristics in the network environment and the novelty of network attacks,this paper proposes the adversarial domain adaptation.The feature between source domain and target domain is aligned by training a target domain feature extractor.The model trained on the source domain can be easily transferred to the unlabeled small volume target domain in this method.Tests on multiple data sets show that the average detection rate and accuracy can be improved by 7.37% and 8.43% respectively compared with model without transfer learning.The deep neural network can learn the feature from the raw data without complex feature project.However,the network flow data cannot be directly input into the neural network.This paper implements a universal network flow processing tool based on byte parsing.It can quickly process network flow data in the form of PCAP into a data format that can be input to the neural network directly.This provides an effective basic method for real-time anomaly detection for network flow data in actual scenarios.