Research On Quality Control Of Meteorological Observations Based On Data Mining

In recent years, with automatic meteorological stations scoped widely in our country, The number of meteorological observation data increase exponentially. The quality of meteorological observation data directly affects the weather forecast and the accuracy of climate prediction.Traditional quality control algorithm cannot satisfy the needs of the quality control work already by only using the historical data value of climate science and elements of allowable values for inspection,lack of sensitivity to abnormal factors’ change. The comparison of data mining method is the trend of hot big data analysis methods in recent years, it was widely used in the meteorological observation data, such as weather forecast and climate predictions, but less used in the quality control of meteorological observation data.This paper introduces an algorithm research on quality control of meteorological observations based on data mining. Puts forward two different methods in quality control of meteorological observation data from the correlation between the same element in different time and the correlation between different elements within the same observation combined with the related algorithm in data mining, and establish a comprehensive quality control plan based on the complementarity and connection within two methods. The main work of this paper detailed as follows.According to chaotic characteristics of the meteorological elements changing over by time, we put forward a method of quality control of meteorological observation data based on time correlation.Firstly, we analysis the chaos characteristics of all the elements in an hour level observation. According to the theory of phase space reconstruction, phase space reconstruction on meteorological observation sequence.Due to the sequence of meteorological data reconstruction has the characteristics of high dimensional and nonlinear, combining the advantages of SVM algorithm.For some time before a period of time data as input, the elements of the elements of the moment data as output, establish the chaotic time series based on SVM forecast model.Aiming at the selection of kernel function in the model, we put forward a hybrid kernel function instead of a single kernel function method, and adopt the improved particle swarm algorithm for determining the parameters in the model, so we can improve the forecast precision of the model.According to the forecast and the actual value curve, we can achieve the meteorological data of measured value of fitting; According to the difference between the observed and estimated values, we can detect the actual measured values of outliers. And with manual error, compared with the traditional quality control method, experimental results show the proposed method to control the quality of higher sensitivity of abnormal data.Secondly, by the correlation between different elements, we put forward a method of quality control of meteorological observation data.The relationship between the elements is difficult to make sure, combined with the advantage of BP neural network, we can know the mapping from input to the output accurately. One of the elements as input, the others as the output, establish the BP neural network based on multi-factor meteorological observation data forecast model.Filtered the input elements by using grey correlation analysis method, delete the strong coupling elements in the input elements, than calculate the remaining elements with output correlation analysis, remove smaller correlation between elements, through the reasonable selection of input elements, we can improve the forecast precision of the model.According to the forecast and the actual value curve, we can achieve the meteorological data of measured value of fitting; According to the difference between the observed and estimated values, we can detect the actual measured values of outliers.Thirdly, According to the complementarity and relevance between the two methods, we establish a comprehensive quality control plan.