Font Size: a A A

Research And Application Of Real-Time Anomaly Detection Technology For Univariate Time Series Data

Posted on:2024-01-29Degree:MasterType:Thesis
Country:ChinaCandidate:Q Y FanFull Text:PDF
GTID:2568307106990099Subject:Electronic information
Abstract/Summary:PDF Full Text Request
With the rapid development and extensive application of Internet of Things(Io T)technology,a large amount of operational data from Io T devices has been generated,including humidity,temperature,voltage,current,and other data.Each data point has a timestamp and is also known as time-series data.Single parameter time series data is referred to as univariate time-eries data.Under the influence of problems such as product defects,performance degradation,and sudden changes in external environment,Io T devices may exhibit abnormal conditions,which are often reflected in single parameter time-series data.Univariate time series data anomaly detection analyzes single parameter time series data with the goal of detecting device anomalies early to avoid greater losses.Therefore,real-time anomaly detection methods for univariate time series data have become a key research issue in the analysis of Io T device operational data.Research shows that statistical methods are one of the mainstream approaches for detecting anomalies in univariate time series data.Among them,the SPOT algorithm(Streaming Peak-over-Threshold),DSPOT algorithm(Drift SPOT),and Flux EV algorithm(Fast and Effective Unsupervised Framework for Time-Series Anomaly Detection)based on extreme value theory utilize online learning to automatically update thresholds.They can effectively identify extreme anomalies in real-time univariate time series data,demonstrating good real-time anomaly detection performance.However,they still have some shortcomings,such as: 1)the maximum likelihood parameter estimation method is time-consuming in the parameter estimation step,resulting in low computational efficiency for automatic threshold calculation;2)the moving average window approach involves frequent data movement operations,affecting the overall realtime efficiency of the algorithm;3)although the moment estimation method is easy to understand and compute,it suffers from inaccurate parameter estimation,thereby affecting the detection accuracy of the algorithm.Based on the above analysis,research is conducted around real-time anomaly detection technology for univariate time series data with the background of the "Residential Central Air Conditioning Cloud Platform" project of a certain company.On this basis,a "Real-Time Monitoring System for Operational Data of Residential Central Air Conditioning" is designed and implemented.The main work is as follows:(1)We propose a real-time anomaly detection algorithm for univariate time series data based on two-parameter estimation,called RTAD-TP(Real-Time Anomaly Detection Algorithm for Univariate Time Series Data Based on Two-Parameter Estimation).It utilizes the Generalized Pareto Distribution(GPD)to fit the excess portion of drifting peaks over the threshold.The algorithm employs a two-parameter estimation method that combines probability-weighted moments with the computationally efficient moment estimation method to improve both anomaly detection accuracy and computational efficiency.Furthermore,a dynamic window scaling mechanism is introduced to enhance the moving average window,avoiding unnecessary data movement operations and thereby improving real-time anomaly detection efficiency.Comparative experiments on two public datasets demonstrate that the proposed algorithm achieves higher efficiency and accuracy.Experimental results on a production dataset from a specific company also indicate that the proposed algorithm performs well.(2)With the large number of devices in the Internet of Things(Io T)system and multiple operating parameters for the same device,there are multiple sources of univariate time series data with different contextual environments.When performing realtime anomaly detection on multiple sources of univariate time series data streams,it is necessary to isolate the contextual environments of the univariate time series data from different sources.To this end,a Memory Concurrency RTAD-TP(MC-RTAD-TP)framework for real-time anomaly detection of multiple sources of univariate time series data based on the RTAD-TP algorithm is proposed.This framework introduces technologies such as thread pool,memory cache,time-series database,and distributed locks to maintain an algorithm instance for each data stream,solving the problem of context crossover caused by concurrent real-time detection.Experiments using a public dataset show that the proposed framework can meet the real-time anomaly detection requirements of multiple sources of univariate time series data.(3)Based on the research mentioned above,a real-time monitoring system for residential central air conditioning operation data was designed and implemented in accordance with software engineering standards,with the background of a company’s "residential central air conditioning cloud platform" project.Through requirements analysis,the problems that the system intends to solve are clearly defined.A microservices-based infrastructure and modular functional architecture were designed to ensure high availability.The previously developed MC-RTAD-TP framework for realtime anomaly detection of single-variable time series data from multiple sources was used to detect anomalies in the single-variable time series data generated by air conditioning equipment during operation.A hybrid data storage solution was adopted to reduce the complexity of storing heterogeneous data from multiple sources.A front-end and backend separation data visualization model was employed to allow the front-end to focus on data visualization while the back-end provides data services.A container orchestration tool was used to quickly deploy all services,providing good performance for real-time monitoring services for residential central air conditioning operation data.The system has been tested and is currently deployed on a company’s online server,operating normally and with stable performance.
Keywords/Search Tags:Univariate time series data, Real-time anomaly detection, Multisource data stream, Data monitoring system
PDF Full Text Request
Related items