Interpretable Anomaly Traffic Detection For Concept Drift

Due to the superior performance provided by deep neural networks and their ability to detect unknown threats,unsupervised deep learning techniques have been widely used in a variety of security-related anomaly detection,so this thesis proposes an anomaly traffic detection model based on autoencoders.However,unsupervised learning has problems with low transparency,poor interpretability,and normal data changes over time during detection,which leads to reduced detection effectiveness and the inability of security operators to trust the model.Therefore,this thesis proposes an explainable anomaly detection model,and strengthens the drift detection and adaptability for the conceptual drift problem in normal traffic data.Aiming at the problem of poor unknown attack detection ability and difficult data labeling of traditional deep learning models,this thesis proposes an anomaly traffic detection model based on sparse autoencoder and an anomaly traffic detection model based on memory-enhanced autoencoder.Experiments on the CICIDS2017 dataset,CIRA-CIC-Do HBrw-2020 dataset and Kyoto-2006+ dataset show that the proposed model shows superior anomaly detection performance in the detection of the above three datasets.Aiming at the problem of poor model interpretability,this thesis proposes an interpreter model based on the abnormal traffic detection model based on sparse autoencoder.First initialize a reference value,and then update the reference value through continuous iteration to obtain a comparison of the outlier and the reference value.Finally,the results generated by the interpreter are analyzed and compared,and the features of various attacks that have the greatest impact on the detection effect are obtained,and the analysis is interpreted according to the features.In this way,the reliability of the model prediction results is further improved.The interpretability of the model is enhanced by interpretable analysis on two datasets,CICIDS2017 and CIRACIC-Do HBrw-2020.Aiming at the problem of drift in normal traffic data,this thesis proposes a drift detection and adaptation model.First,a confidence calibration component is proposed,which guarantees the agreement between the probability values predicted by the model and the actual observations.Then this thesis proposes a drift detection method,which detects drift by calculating the difference before and after drift.In addition,an interpreter is designed to help security operators determine and understand drift by optimizing the problem to find important samples of normal data drift,thereby reducing labeling overhead.Finally,a special regularization term is added to the existing incremental adaptive method to prevent catastrophic forgetting.Experiments on the Kyoto-2006+ dataset show that the model in this thesis can not only detect drift better,but also has a stronger ability to adapt to drift.