Data mining applications for updating missing values of traffic counts

Estimating missing values of data records is known as data imputation. Highway agencies have traditionally used simple factor and time series analysis models to impute missing values since traffic data programs were established in 1930s. Literature review shows that current practices for imputing traffic data are varied and intuitive in nature. No research has been conducted to assess the imputation accuracy. In this thesis, various data mining techniques are applied to two large traffic databases from Alberta Transportation and Minnesota Department of Transportation to study the nature of missing values and data imputation. The imputation practices are reviewed, and traditional imputation models are statistically evaluated. Improved traditional models and advanced models based on modern techniques are developed for more accurate imputations.; Models based on advanced techniques, such as genetic algorithms (GAs), time delay neural networks (TDNN), locally weighted regression (LWR), and minimum square error (MSE) models, are innovatively developed for estimating missing data from traffic counts. MSE models are used to match traffic pattern having missing values with those without missing values for imputations. Regression and neural network models are developed by using data from before the failure, or data from both before and after the failure as candidate inputs. GAs are used to select final inputs from the candidate inputs for both regression and neural network models. It is found that models using data from both before and after the failure outperform those using data only from before the failure. Study results clearly indicate that the advanced imputation models have much higher accuracy than traditional models. For example, for the same commuter count cited previously for testing the traditional models, the 95th percentile errors for the MSE model are usually less than 6%. For neural network models using data from both before and after the failure, the 95th percentile errors range from 7% to 9%. For regression models using data from both before and after failure, the 95th percentile errors are less than 1.5%. It is believed that highway agencies could use the proposed models to significantly improve the quality and cost-effectiveness of their traffic data programs. (Abstract shortened by UMI.)...