Research On Key Technologies Of Magnetic Flux Leakage Data Cleaning For Pipeline Internal Detection

In the most critical defects inversion problem of inside testing technology for pipeline magnetic flux leakage,a large number of algorithms and models are based on the ideal data set.However,in the operation of the actual project,due to the complexity of working environment of detector device,as the extension of running time and distance,transient fault is used to appear unavoidably on individual sensor when collecting magnetic flux leakage signals,so there are a lot of abnormal points of the resulting data sets.These anomalies will seriously affect the quality of the data mining and become an obstacle to the subsequent data analysis work.Therefore,in order to improve the reliability of analysis results,it is necessary to do data preprocessing work before the analysis of magnetic flux leakage data set,data cleaning is an important link of this pretreatment.This thesis introduces the background and significance of this selected topic,and put forward two major problems that abnormal data cleaning technology need to solve,which are the removal of the abnormal point problem and data missing interpolation problem.This thesis will separately carry on the detailed introduction and analysis.Concrete research content is as follows:Firstly,an anomaly detection method for magnetic flux leakage testing data is designed.First of all,analyzing the specific characteristics of leakage magnetic anomaly data and complete the exception type classification.Then according to different exception types,a corresponding abnormal remove method is designed,and eliminate effect of an abnormal data set point will be analyzed.Secondly,a magnetic flux leakage of missing data interpolation method is designed.First of all,analyzing missing data sets with the abnormal points removed and classifies it according to the degree of losing.Use bilinear interpolation method to deal with the types of random single point defect.In view of the lack of regional data types,combining three times though laser interpolation and cubic spline interpolation,an adaptive weights for multiple interpolation method is designed.Through the analysis of the simulation and test results,complete the evaluation of the merits of the algorithm.Thirdly,an interpolation method of missing data based on KNN(K-NearestNeighbor)of magnetic flux leakage is designed.In order to improve the lack of regional types of interpolation precision,through processing a complete history of magnetic flux leakage signal data characteristics,build KNN neighbor search sample set,using the corresponding data of neighboring data points to predict for the absence of interpolation sample data points.In order to overcome shortcomings of KNN algorithm of the low efficiency during searching period,through the combination of K-D tree(K-Dimensional tree)data structure and improve nearest neighbor search perimeter calculation method,design the KNN efficient neighbor search method based on K-D tree in high dimensional space.Fourth,a precise interpolation method based on KNN and SVR(Support Vector Regression)is designed.According to the corresponding position in the interpolation samples,whether missing or not,divide the samples of each neighbor into two parts as input and output,using the SVR to regression,the model was applied to the interpolation sample;complete the missing point interpolation tasks.Through the analysis of interpolation results,design data interpolation method which can cope with different situations process under the different requirements.Last but not least,on the basis of summarizing the full thesis,the future research direction is also prospected.