Evaluation Of Prediction Performance For Crash Injury Severity Between Statistical And Machine Learning Models

Road traffic accident data analysis is one of the prime interests in the present era.Analysis of accident is very essential because it can expose the relationship between the different types of contributing factors that commit to a road accident.In addition,the predicting accuracy of crash risk models needs to be further improved.Nowadays,Data mining is a popular technique for examining efficiently the accident dataset.In this work,OP and MNL as two Statistical and CART,SVM,KNN,GNB and RF as machine learning classification models have been implemented on the dataset of the road traffic accidents.It provides an opportunity to explore new models with more powerful performances.This study evaluated the predictive performance for crash injury severity between various machine learning and statistical models with distinct modeling logic.Based on crash data collected for different districts councils of Hong Kong,the models are applied for predicting the injury severity associated with each crash severity level.The predicting accuracy of each model on testing set is calculated and compared.Then the sensitivity analysis is performed to infer the importance of explanatory variables on crash severity.Sensitivity analysis assesses how “sensitive” the model is to fluctuations in the variables and data on which it is built.The significant variables collected from OP and MNL statistical models are same and are used for sensitivity analysis purpose.From the sensitivity analysis results,we can make the suppositions that the five selected machine-learning models have considered the order of crash severities.The results showed that machine-learning models had higher predicting accuracy than statistical methods,though they suffered from the over-fitting issue.The fatal accidents classification accuracy by RF,GNB,KNN,SVM and CART is 82.77%,55.53%,82.82%,77.93%,and 81.64% respectively.In particular,the CART and SVM were found to be the best machine learning models that had the highest overall predicting accuracy,which were 85.78% and 84.24% respectively.