Driving Fatigue Detection Based On Subtle Facial Movement Recognition And Deep Learning

With the development of traffic,vehicles have provided remarkable convenience to people’s travels,property losses and casualties caused by traffic accidents also increase yearly.Research and investigations have shown that fatigue driving is an important factor in traffic accidents.Therefore,the effective detection of fatigue driving behavior has become a research hotspot in the field of transportation.Yawn and doze detections are combined in this study to perform fatigue driving detection.Facial movements include mouth and eye movements.The movement of the face is relatively small compared with that of the body and can be regarded as subtle movement.This research includes the recognition of two subtle facial movements,namely,mouth and blink movements.Yawn detection based on subtle mouth movement recognition.Many existing yawn detection methods are based on a single static picture,and few methods recognize yawn as a facial movement.Many facial movements and expressions have mouth open states similar to yawn.If yawn is considered as a static state for recognition and counting,then false detection will easily occur.Some methods use the mouth opening time as a judgment condition,but these methods need to judge the mouth state of each frame in the video,and the calculation is expensive.In order to solve the above problems in yawn detection.Yawn is used as a facial movement recognition method to detect,and a yawn detection method based on subtle movement recognition of the mouth is proposed.This method does not need to process every frame in the video,which greatly reduces the time consuming of detection.Following methods are used in this study in order to recognize yawn effectively.(1)An efficient and fast keyframe selection algorithm is improved to extract representative frame sequences in the mouth movement video sequence.These keyframes are applied to the proposed recognition model for subtle mouth movement.(2)A subtle movement recognition model of mouth is constructed(3D-LTS).This model is based on 3D convolutional and bidirectional LSTM networks,which can extract long-term movement features.The sampling characteristics used in this study can improve the accuracy of subtle mouth movement recognition,and yawn can be effectively detected through subtle mouth movement recognition.Doze detection based on eyes state recognition.Doze is a subtle movement and state of the eye.Eye state recognition in traditional doze detection is based on geometric features,such as the distance between the upper and lower eyelids,and relies on the positioning of the key points of the eyes.However,the recognition efficiency of these traditional methods considerably varies under different distances,illumination,and occlusion environments.Several existing methods based on deep learning are time consuming and have many model parameters.In order to solve the above problems in doze detection.A densely connected multi-pool convolutional network model(DMP-Net)is proposed in this study to detect the eye state.This study detects blinking movements via changes in open and closed eyes and performs doze detection through blinking frequency and PERCLOS.DMP-Net uses the dense connection structure of DenseNet.This model enables the network to obtain enhanced performance with high real-time performance.Color and infrared images are used as input in the experiments,and this condition can effectively improve the detection effect in the case of eye occlusion.This study has carried out experiments on the improved algorithm and the proposed model,and some of the results have been published in the top journal of SCI.The yawn detection method proposed in this paper effectively reduces the number of false detections.The proposed doze detection method can detect the driver’s dozing behavior under different eye obstructions.Comparative experiments with other fatigue driving detection methods prove that the method in this paper can quickly and efficiently detect the driver’s fatigue driving behavior.