Research On Weakly-Supervised Anomaly Detection Algorithm Based On Deep Autoencoder

Anomaly detection related technologies have been well explored in a great deal of application scenarios.However,in actual production and life,because there are often no or only a few abnormal samples in the data set,data over-fitting problems often occur,and it is more likely to be seriously affected by noise and large outliers,which promotes the research of weakly supervised anomaly detection algorithm.Based on deep learning,this thesis proposes a new weakly supervised anomaly detection algorithm model based on autoencoder.The main research contents are as follows:(1)A series of methods have been adopted to enhance the robustness of the anomaly detection algorithm model,including selecting the original data set in different fields,filling the missing data in the original data set,normalization preprocessing and adding noise to the training set.And a series of feature selection comparative experiments are carried out to explore the existence of useless features,such as noise and redundant information.The experimental results show that all features are useful for classification,and the classification effect of the original dimension data is the best.(2)A new anomaly detection algorithm model is proposed:the coding part is based on four-layer long short-term memory network,the decoding part is based on the decoding block composed of three-layer one-dimensional convolution and deconvolution.Among them,the residual connection structure is formed by adding the feature vectors outputted from the network of the first three layers of the coding layer and the outputs of each layer of the decoding section,which is used the channel attention vector of the feature vector.(3)A new joint training strategy is proposed to further solve the problem of data over-fitting:linear regression error and reconstruction error are combined into joint error.And comparative experiments are carried out from three dimensions:different data processing methods,different module settings and different training methods.The four system test performance indicators all perform well,which reasonably verifies the progressiveness and robustness of the algorithm model designed in the thesis.The research in this thesis combines and utilizes emerging technologies related to deep learning.Based on the research and improvement of deep autoencoder networks,a weakly supervised anomaly detection algorithm model is designed and constructed,achieving effective detection of anomaly data from multiple different industries across different fields,which has important theoretic value and practical significance.