The Applied Research Of Data Mining Techniques In The Problem Of Road Impedance Function

With the advent of the era of big data, the development of data warehouse and data mining technology have become rapid, using the circulated and precipitated data of existing systems and digging out the useful model to guide decision-making, has become a trend in the era of big data. With the increase of emergencies and natural disasters, the emergency logistics for finished product of grain is increasingly valued by government and researchers, in order to build a scientific, efficient, and reliable emergency scheduling and decision system for finished product of grain, this paper proposed the research idea to use the data mining technology to solve the distribution route optimization problem in emergency scheduling of the finished product of grain.In this paper, the main algorithms and evaluation indicators in data mining technology has been studied, the advantages and disadvantages of each algorithm has also been analyzed and summarized. Trough combing data mining technology with the emergency scheduling and decision of the finished product of grain, and in order to solve the dynamic path optimization problem, this paper proposed to use the regression analysis technique which belongs to data mining technology to study the road impedance function.First, in this paper, our study is based on actual data. As we collected the microwave detection road data of Beijing in January2012, we used the distributed data processing means to clean and process the historical road data. And we studied the data storage and processing principles of the distributed system which named hadoop. In order to facilitate the follow-up model study, we proposed the idea of feature extraction under the large-scale data sets, and designed the feature extraction step for the study of road impedance function and also gave a simple example.Then, in the part of model study, we used data mining techniques of classification, regression techniques to fit the target values and input features, thus to determine the road impedance function. This part of research is divided into four phase, we study the impedance function from simple means to complex ones:Linear model, BPR-based function model, classification and regression tree model, and then we put forward the probabilistic classification and regression model Innovatively and an in-depth study is given. We also gave the detailed derivation, solving methods, advantages and disadvantages of each one of these four types of models which are presented in this paper.Finally, in the part of model validation, we designed detailed experimental procedures for each of the four models, then trough the experiments which based on the actual historical road data of Beijing in January2012, we compared, analyzed and verified the four models’ experimental results using quantization indexes. Through the experiments we can prove that the probabilistic classification and regression model presented by this paper has the best performance and practical value under the problem of fitting the road impedance function.