Research On Driver Fatigue Monitoring Algorithm Based On Multi-task Cascaded Convolutional Neural Network

Fatigue driving is one of the main reasons which lead to traffic accidents.Driver fatigue monitoring system can identify the state of driver and warn driver of dangerous driver according to the change of driver behavior,by which the number of traffic accidents can be reduced significantly.Therefore,driver fatigue monitoring system is one of research hotspots in terms of active automobile safety.The development of convolutional neural network,which is another research hotspot,improve the accuracy of many different visual tasks drastically.This thesis is aimed at solving the problem that the accuracy and robustness of state-of-the-art driver fatigue monitoring methods are dissatisfied by proposing a novel multi-task cascaded convolutional neural network and a driver fatigue state identification method based on discipline-crossing,so that the fatigue features can be extracted precisely and the system can make accurately judgement.The main research contents of this thesis are shown as following :1)Feed-forward convolutional neural network optimizing.In order to make feedforward network more suitable to driver fatigue features extraction,this thesis optimized VGG16,VGG19,Resnet50 and Resnet101,four classical networks.The real-time capability of the networks are improving without performance degeneration by changing the size of input image and red ucing the number of convolutional layers.The activate layers are optimized and every convolutional layers take the advantages of Batch Normalization so that the network can achieve the same accuracy with fewer training steps.2)Multi-task convolutional neural network(CNN)designing.In order to improve the accuracy of feed-forward network further,this thesis proposed three novel components of CNN: task simplifier,heatmap generator and feature map generator.Then,these three novel components are combined with the optimized feed-forward neural network to build the multi-task cascaded CNN which can achieve face tracking,head pose estimation and facial landmark localization at the same time so that the fatigue features can be extracted accurately.3)Driver fatigue state identification algorithm designing.This thesis proposed a driver fatigue state identification method based on Long Short-Term Memory which can utilize multi fatigue features in a video sequence to evaluate the state of driver comprehensively so that the accuracy of driver state identification can be improved significantly.4)Driver fatigue monitoring algorithm evaluation.This thesis evaluated the four kinds of optimized convolutional neural network and chose the CNN with best performance as the feed-forward part of multi-task CNN.Then,the three novel components are evaluated on benchmark dataset and the results show they can improve the accuracy of multi-task CNN significantly.The multi-task CNN is evaluated on both the benchmark datasets and the datasets in real scenarios and analyzed quantitatively according to the results.Finally,this thesis built up a dataset for fatigue driving monitoring on a virtual driving platform and the driver fatigue state identification is evaluated on the dataset.In the multi-task CNN,the errors of facial landmark localization,head pose estimation are 6.20%,1.4° respectively and and precision of driver face validation is 98.3%.The accuracy of driver fatigue state identification method reaches 93.7% in the experiment.The results illustrate the proposed driver fatigue monitoring method obtains highest accuracy and is the most suitable method on driving condition compared with other state-of-the art methods.