Research On Driver Fatigue Detection Based On Self-supervised Learning And Graph Attention Mechanism

With the rapid development of society and economy,the number of motor vehicles is increasing,which aggravates traffic congestion in cities and causes many traffic accidents,among which driver fatigue is one of the main causes of traffic accidents.The driver fatigue detection methods based on computer vision is an important research direction in the field of computer vision and pattern recognition.Existing driver fatigue detection methods suffer from insufficient robustness to image noise,failure to fully leverage the interrelationship between sequence features,and difficulty in effectively distinguishing peak frames from non-peak frames.In this thesis,we address the above problems and propose a Self-supervised Multi-granularity Graph Attention Network(SMGA-Net).The main contribution of the method are as follows:(1)An Image Restoration based Self-supervised Learning(IRS-Learning)is proposed to improve the robustness of the network to noisy input images.The method adds random noise to the input image and restores the features extracted from the noisy image by a Convolutional Neural Network(CNN)to a noise-free input image by a Decoder.The image restoration learning process optimizes the parameters of CNN to improve its ability to learn effective features from noisy images,thus achieving robustness of SMGA-Net processing with noisy input images.Additionally,a hybrid loss function combining cross-entropy loss and structural similarity loss is constructed to achieve multi-task learning of image restoration and fatigue driving detection.(2)In order to achieve the fusion of multi-granularity features in SMGA-Net,a Cross Attention based Feature Fusion(CAF-Fusion)method is constructed,which takes the multi-granularity features extracted from the face image by the convolutional neural network as input,learns the coefficients corresponding to the information in each part of the multi-granularity features through the cross attention mechanism,and performs feature fusion according to the coefficients,therefore the effective information in the fused features is highlighted and the invalid information is suppressed.(3)In order to achieve the extraction and utilization of interrelationships between sequence features by SMGA-Net and the effective differentiation of peak frames from non-peak frames,this thesis constructs the Multi-head Graph Attention(MHGAttention)method based on the graph attention mechanism.The method takes multigranularity fusion feature sequence as input,constructs interrelationships among sequence features through the graph attention mechanism,and realizes information sharing among sequence features;updates sequence feature information based on the learned interrelationships and shared information,identifies the different importance of each sequence feature to the final classification,and generates corresponding importance weights to achieve the distinction between peak frames and non-peak frames.The validation experiments of the proposed methods are conducted on the NTHUDDD dataset,and the experimental results are compared with related works to verify the effectiveness of the proposed methods.