Research On Traffic Flow Prediction Based On Spark Open Source Framework

Traffic flow prediction of urban roads is helpful for relevant authorities to coordinate traffic supply and demand and formulate traffic management plans.Meanwhile,it provides useful reference for travelers to plan their routes.What’s more,it is a core component of research and application of ITS(Intelligent Transportation System).Nevertheless,as a complex subject,traffic flow prediction has undergone tremendous evolution,from the mathematical statistical analysis of traffic characteristic modeling to prediction based on the intelligent prediction algorithm discovered by virtue of knowledge.Based on large-scale data and with an aim to provide accurate predictions,the intelligent prediction algorithm takes in-depth learning as the core and has now been applied to the field of traffic flow prediction.In order to solve problems regarding the precision of traffic flow prediction and the processing of mass data,this paper proposes a combined prediction algorithm with higher precision and builds a big data processing platform for traffic flow based on spark open source framework.The main research contributions of this paper are as follows:Firstly,the K-means clustering algorithm is adopted to obtain the information implicit in traffic information flow,so as to solve the difficulty in characteristic extraction resulting from the lack of information characteristics in traffic flow.On this basis,improvements have been made in two aspects: the need of manual input to determine the number of initial clusters using the traditional K-means algorithm;random selection of central vectors of initial clusters.Experiments on six artificial datasets indicate that the improved K-means algorithm has achieved a higher score in the cluster evaluation index.Secondly,using the traffic indexes of Xi’an in 2018 provided by Gaia Open Dataset,this paper proposes an algorithm based on data of similar historical moments,removing and filling data missing values and non-compliance data,and adopts the characteristic normalization algorithm to accelerate the convergence iteration of the model.Next,the paper puts forward a K-means-LSTM algorithm by combining the improved K-means algorithm with LSTM(Long Short-Term Memory).This algorithm can predict the average interval speed over the next day.The experimental results suggest that the combined model compares favorably with the application of LSTM alone in various model evaluation indexes.For instance,an increase of0.0884 in R-squared makes it more superior than other algorithms such as SVR and GRU.Finally,considering the fact that the time series prediction algorithm can only predict limited road conditions,this paper adopts a data processing platform for traffic flow based on spark open source framework.The platform has realized the following three functions as well as their real-time visual display: predicting the average interval speed of 793 roads in Xi’an simultaneously;monitoring traffic dynamic information;and making statistics on congestion.