The Design And Implementation Of Anomaly Detection System In Cloud Environment Based On LSTM

In recent years,with the continuous increase in the scale and complexity of computer applications,enterprise IT systems have gradually migrated to cloud platforms such as public clouds or private clouds.Timely and effective diagnosis and warning of system failures play a key role in avoiding economic losses caused by system downtime.However,the traditional anomaly detection system based on a static threshold has the defects of a high false alarm rate and low accuracy rate.Therefore,to improve the effectiveness of anomaly detection,this thesis proposes an unsupervised anomaly detection model based on LSTM and builds a cloud-oriented anomaly detection system.The main research contents of this thesis are as follows:(1)For the multivariate time series anomaly detection problem,QA-LSTM,an anomaly detection model based on LSTM is proposed.The model adopts the dimensional attention mechanism,which is different from the traditional way of applying attention on time steps,and instead weights the relevant variables so that the value predicted by learning the time dependence of the series and the correlation between the series at the same time.For the traditional error-based fault diagnosis method,quantile regression is used to predict the confidence interval to achieve single-metric anomaly judgment,and the metric majority voting mechanism is used to calculate the anomaly score,which improves the anomaly detection performance in a more fine-grained way.After testing on public datasets,the algorithm proposed in this thesis outperforms benchmark models in F1 score,effectively learning the data features in high-dimensional datasets.(2)A deep learning-based anomaly detection system is designed and implemented in this thesis.The system collects information on the monitored equipment in real-time by deploying an agent program on the facility to be monitored.The service discovery mechanism is introduced to achieve the tracking of the device’s online status.Anomaly detector integrates the algorithm proposed in this thesis and provides services such as model training and online detection in the form of web application.The monitor server implements real-time data collection and alarm management,interacts with the anomaly detector in the form of timed tasks,and checks the relationship between the current state and the alarm strategy to achieve real-time anomaly monitoring.In addition,a data visualization module is designed and implemented in this thesis,which visualizes real-time data and alert configuration.(3)The system modules including the data access,anomaly detection,anomaly alarm,and data visualization are verified in this thesis.The tests show the high availability and effectiveness of the system.Based on the running data of real enterprise servers,the actual performance of the algorithm proposed in this thesis is verified.The low training cost of the algorithm enables the model to be rapidly deployed and iterated in time.In summary,an anomaly detection system for the servers in cloud environment is designed and implemented in this thesis.The system integrates an algorithm named QA-LSTM proposed in this thesis.The systematic test verifies that the modules in the system operate stably and are able to assist enterprises in automatic operation analysis.