Research Of The Short-term Traffic Flow Prediction Based On Spark Platform

With the increment of vehicle,urban traffic congestion has become even worse,which has seriously affected people’s daily life. Through real-time and accurate traffic flow forecasting can provide the basis for traffic management and guidance,and thus effectively solve the problem of urban traffic congestion. Traffic flow prediction is to find the inherent law of traffic flow and to realize the real-time and accurate prediction of the road traffic flow through mining the traffic flow data from the data acquisition equipment. And with the development of data acquisition and data acquisition equipment widely used,the traffic flow data has dramaticly increased,how to effectively process and analysis the sharply increased traffic data,has become a reasearch hotspot in recent years.The traditional data processing mode based on single physical machine can not meet the requirements of massive data storage and processing due to the limitations of memory,disk,CPU and other factors. In order to slove the performace bottleneck of the single environment in dealing with massive data,we put forward a short-term traffic flow forecasting method based on the Spark distributed computing platform relied on the advantage of big data computing framework in processing massive data,this method effectively improves the computation efficiency in ensuring the prediction accuracy, and has strengthen the practicability of the prediction algorithm.The major work in the thesis includes:Firstly, Based on the space-time characteristics of traffic flow,we proposed the traffic flow feature vector generation method,and constructed the short-term traffic flow forecasting model based on the space-time relation.Secondly, Because the single physical machine has low calculation ability and low expansion in processing massive data, we proposed the KNN algorithm parallel implementation method in spark platform,this method effectively sloves the low efficiency of the KNN algorithm for nearest neighbor search in dealing with history database,and applied the parallel KNN algorithm in the short-term traffic flow forecasting,under the precondition of ensuring the prediction accuracy,this method improves the computation efficiency of the algorithm and improves the practicability of the KNN algorithm.Finally, We select the actual traffic flow data from California PeMS system as the experimental data. Used the accelerate ratio、root mean square error as the evaluation index,and used the distributed cluster composed of multiple nodes for simulation and prediction. The experimental results show that the short-term traffic flow forecasting based on Spark platform can meet the requirements of the massive traffic data processing,and the system has good scalability.