Driver Behavior Analysis Based On Facial Landmark Information

With the increase of vehicle ownership,vehicle safety has become a more and more important issue.Most of disastrous traffic accidents are caused by the fatigue driving every year.Moreover,the smoking driving behavior is not included in the traffic regulation,but there are still some traffic accidents that is actually related to the smoking driving.Therefore,the monitoring of operating vehicles,such as "two passengers and one danger" vehicles,express vehicles,as well as online booking vehicles is particularly urgent and important.For detecting the fatigue and smoking driving of drivers,there are two main methods.The first one is the fatigue detection based on the physiological characteristics of drivers.The fatigue detection is accurate,but the portability is poor because the detection devices must contact with the driver physically.The second one is the fatigue detection based visual information,which is fast and easy to deploy.Traditional methods of feature extraction require high quality images as input and heavily rely on handcraft designs,which limits the variety of feature.For the smoking detection,the commonly used methods are to check the existence of smoke in the cab by special devices,or to measure the temperature of the cab.However,such smoking detection methods are sensitive to the surrounding environment,which will increase the rate of the omission and false detection.This paper proposes a method for fatigue and smoking driving detection.For the fatigue driving detection,we adopt the facial landmark information to determine whether the driver is fatigue driving.For the smoking detection,we synthesize smoking images with the information of the mouth to solve the problem of lacking training data.Our main contributions can be summarized as:(1)We train a model named CNN-9 for the facial landmark detection and use it to extract the facial landmarks of drivers for the fatigue driving detection.Our trained model is compact,fast and achieves high performance on testing set.Besides,the experimental results of fatigue driving detection on 109 videos show our proposed method is accurate.(2)We model the smoking detection as a binary classification problem and design a model named VGGNet-10 for the smoking driving detection.In addition,massive smoking samples are synthesized through the landmarks information to solve the problem of lacking training data.(3)To solve the imbalance caused by the synthetic training samples,we propose a fine-tuning strategy: the mode is firstly trained on the synthetic samples,then is fine-tuned on the real data.The proposed fine-tuning strategy can achieve better accuracy on the real testing set of smoking.Therefore,our smoking detection model,smoking images synthesis and fine-tuning strategy are effective.Besides,the fast running speed in CPU ensures the practicability of our model.In summary,we provide a model for landmarks detection of drivers,and then build an evaluation metric for the fatigue driving.Finally,we achieve both the fast running speed and high accuracy of the fatigue driving on videos.In addition,we train a model for smoking detection through a fine-tuning strategy,and this model reaches a high recall rate on testing set and fast running speed for practicability.