Research On Urban Road Congestion Prediction Based On Spark

With the continuous progress of social economy,people’s living standard has gradually improved,and the number of urban private cars has also steadily increased year by year.The unbalanced relationship between traffic demand and the limited capacity of roads caused by the increasing number of motor vehicles has become more and more serious,and finally appeared on the roads of various cities in our country in the form of traffic congestion.At present,the problem of traffic congestion is becoming more and more serious in many urban roads in our country.Frequent congestion not only causes travel time delays,but also aggravates vehicle pollution emissions.The key to the management of urban road congestion is to predict possible traffic congestion in advance,to issue early warning signals to various congestion situations,and to take corresponding measures in advance.Therefore,timely prediction of the occurrence of congestion and early warning of various possible congestion is very important to the development of urban transportation in our country,which is of great significance to alleviating traffic pressure,improving travel safety,saving travel time,and reducing energy waste.Based on the urban road traffic data,study the prediction method of urban road congestion.The main work includes:consulting related literature,studying the development status of road traffic congestion prediction algorithms,parallelization technology,and big data cloud computing technology.Summarizing the existing problems and shortcomings of existing research,determine the optimization random forest based on Spark parallel as the research goal.Research related big data technology and build the cluster environment.Research from the angles of data processing,feature selection,parameter optimization and so on.Using threshold method and traffic flow mechanism method to distinguish abnormal data and missing data,and using moving average method and historical trend method to process missing and abnormal data.Build theχ2-YYPO-RF model,using the chi-square test to perform feature selection on the processed data,and optimizing the parameters of the random forest through the Yin-Yang-Pair optimization algorithm,and reasonably set the number of ntree and mtry,which are two key parameters affecting the prediction.And base the Spark big data platform design the model parallelization.In order to determine the feasibility of the research content,through the accuracy experiment,compare the K-nearest neighbor algorithm,the long-short-term memory neural network algorithm,the traditional random forest algorithm and theχ~2-YYPO-RF model in terms of accuracy,precision,recall and F1 measurement.Through speedup experiment and scalability experiment,compare the speedup ratio and expansion ratio of the parallelized model under different number of nodes and different data volume to verify the effect of model parallelization.The experiments show that the proposedχ~2-YYPO-RF urban road congestion prediction model has a prediction accuracy of 95.58%,and the four evaluation indexes are all higher than other models.With the increase of the number of nodes and the amount of data,the acceleration ratio of model gradually increases,the expansion ratio gradually decreases,and gradually approaches the ideal value.