Driver’s Activity Recognition Based On Computer Vision

As automobile improving,the quantity of cars is increasing fast.Road accidents largely affect the family life and social stability.The driver’s behavior largely affects the road traffic safety.Effective recognizing driver behavior and timely warning can greatly avoid the occurrence of accidents,so as to ensure the safety of drivers.In addition,in the middle and low levels of autonomous driving vehicles,the driver should take control of the vehicle in a timely manner when the autonomous driving system fails.Real-time monitoring of driver behavior is of great benefit to the assessment of driver take-over ability and the design of complete autonomous driving decision-making algorithm.Therefore,it is of great significance to carry out the research on driver behavior recognition algorithm.For natural driving scenarios camera image noise is big,the data category sample is not balanced,the existing algorithm of time and space problem such as poor ability of feature extraction,this paper studies the based on the data of the human body skeleton key point,using the method of deep learning fully fusion,the space-time characteristics both accuracy and real-time performance of driver behavior recognition method.The main contents are as follows,(1)Aiming at the problem that most of the existing open-source data sets are picture data sets and lack of continuous sequence context information,this paper constructs a video data set of driver behavior.The data set consists of two parts: natural driving data and simulated driving data.According to the natural driving conditions,the process of simulating driving data collection was designed.On the premise of ensuring the reliability of the simulated data,the number of samples for the second driving task was increased.Alphapose algorithm is used to obtain the key points of driver skeleton.Using human skeleton to represent human poseture can reduce the background noise.Through the transfer learning method and the temporal exponential average filtering algorithm,the performance of the Alphapose algorithm is improved.And the refined skeleton key points provide the data basis for the driver behavior classification algorithm.(2)A driver behavior recognition algorithm based on LSTM neural network is proposed.A simple single-layer LSTM network was constructed,and the set of driver skeleton key points at each moment was taken as the input of the network,and the temporal dependence of skeleton key points sequence was established.The LSTM network is trained and tested on the video data of driver behavior.The results show that the LSTM network can effectively converge and basically solve the problem of driver behavior identification,but the identification accuracy needs to be further improved.This network will be used as the baseline model of this paper and as the basis for comparison of subsequent optimized networks.(3)A driver behavior recognition algorithm based on spatio-temporal graph convolutional neural network is proposed.LSTM network only takes the skeleton key points set as input and lacks the key points connection relation,so it is difficult to fully express the spatial structure characteristics of human body posture.Therefore,we propose the graph convolutional recurrent neural network.According to the human body structure,the driver skeleton graph is designed,and a five-layer GCN is constructed to extract the spatial structure features of driver behavior.Then,the spatial structure features of each frame are input into the LSTM network to extract the temporal continuous features of the skeleton sequence.We utilize the attention mechanism to integrate the characteristics of every timestamp to obtain the comprehensive spatio-temporal characteristics of driver behavior.Aiming at the problem of data imbalance of driver behavior data set,Focal Loss Function was introduced to adaptively balance the loss values of different categories of data and optimize the network training process.In this paper,sufficient ablation studies and cross validation experiments are designed.The results show that the spatio-tempoal graph convolutional neural network has a greater improvement compared with the baseline model by introducing graph convolutional LSTM network and attention mechanism module.In addition,the evaluation results on the same data set with the existing models show that the spatio-temporal graph convolutional LSTM network has better identification accuracy.In the real-time application experiment,the spatiotemporal convolutional LSTM network also has better real-time performance.