Application Research On Data Cleaning Technology For Tailings Reservoir Monitoring System

In recent years,with the construction of mineral enterprise information technology,a large amount of basic data has been collected in the enterprise information system,and the use of these data for decision analysis has extremely practical significance.Due to the inaccuracy of the data,abnormal or missing data will be generated,which seriously affects the data quality.Therefore,using data cleaning techniques to improve data quality,optimizing data sources is an indispensable step.The tailings pond monitoring system first uses sensor equipment to collect data,and then analyzes the data to achieve safety monitoring of the tailings pond.Because the tailings pond is affected by the external environment,the data collected by the equipment will produce abnormalities and missing data.Firstly,the abnormal data should be detected,and then the missing data can be corrected and filled according to the trend before and after the observable data.Improve data quality.The main research work of this is as follows:Firstly,based on the spatio-temporal correlation of the perceptual data,the data collected by each device in the tailings pond system is studied,and the patterns are classified according to the trend of the data before and after,and are classified into “individual abnormal points” and “abnormalities”."Sequence" two major categories divided into two categories: "individual anomaly point" and "abnormal sequence".Then,based on the similarity and representation of the anomaly data,a neighboring difference jump anomaly detection algorithm is proposed.The algorithm draws the idea of interval and statistics in mathematics and the idea of discriminant analysis in data analysis theory.The threshold value of the neighbor difference is used to judge whether the data collected by various sensing devices is abnormal,and to find out abnormal data of different categories.It is also compared with statistical methods and neighborhood density based anomaly detection algorithms.Finally,the experiment selected the data collected by the recent tailings pond immersion line equipment.First,experiments were carried out using the above three abnormality detection algorithms.After analyzing the experimental results,it is found that the accuracy of finding abnormal data on the collected data of the tailings pond using the nearest neighbor difference hopping algorithm is more significant than the statistical method and the neighborhood density algorithm;then,using the above three kinds of padding algorithms,the abnormality is found.After the value is filled,the experimental results show that the improved weight-based neighbor algorithm has a higher degree of fit in filling the tailings anomaly and missing data,and the applicability is better.