Inspection And Analysis Of Highway Transport Freight Market Using Network Crawler Technology

With the rapid development of China's economy,the freight market continue to grow and the volume of road,railway,waterway and other freight is also increasing.However,freight volume information is difficult to monitor,because it has a wide range of data sources and needs to aggregate,filter and store a large number of scattered information in the database.Therefore,this paper used the network crawler technology to obtain and summarize the traffic data of highway Freight market,the quadratic curve model to repair the missing data,and the exponential smoothing model to forecast the future traffic trend based on historical data.Firstly,this paper used the Web crawler-Octoparse instruments to obtain the traffic information of freight market from different websites,and subdivides it into weekly,monthly,seasonal,annual,mode of transportation,region and volume of freight.Regarding region as a unit,it summarized the traffic data of different time periods under the same mode of transportation,and explored all missing values in the data.Then it took the freight volume information of North China,Shandong and Central China as an example to repair the data.It chose other data of the year in which the missing data is located,and use quadratic curve model,exponential model,logarithmic model to fit the curve.According to the R-square values of different models,selected the best fitting model,determined the model parameters and calculated the missing values,and also repaired all missing values in turn.Finally,based on the data of 2016,2017 and 2018,the data of the first 22 weeks of 2019 are predicted.The Exponential smoothing model is used to establish the forecasting model of future freight volume in the three regions.Considering the obvious seasonality of the data,the results of the forecasting value and the actual value under three different seasonal conditions are compared by using the three-time exponential smoothing method(Holt winter exponential smoothing method).Finally,the prediction results show that under simple seasonal conditions,the prediction effect is the best,which is close to the actual value.