Driver Gaze Prediction Based On Feature Fusion

With the increasing popularity of unmanned driving,the assisted driving system,as the upstream of the automatic driving predictive control chain,has received extensive research and attention in recent years.The driver’s gaze prediction is the core technology in the assisted driving system.By detecting the driver’s gaze direction,the driving system can assist the driver to make corresponding actions,so that the driver’s driving process is more comfortable,safe and stable.Compared with the limitations of manual features in the past,with the rapid development of deep learning in recent years,the feature extraction method based on deep neural network has higher accuracy and stronger generalization ability,so it has begun to be widely used in driver gaze prediction task.The main work of this thesis is as follows:(1)An appearance-based feature extraction model is proposed,which is mainly divided into a global face model and a local binocular model.For the global face model,the mainstream CNN network is used for feature extraction,and the 3D supervision signal is added to strengthen the model’s feature learning of the head pose,thereby improving the gaze prediction accuracy;for the local binocular model,a branch network is designed to detect multiple face regions.The feature extraction and feature fusion can strengthen the connection between local areas.(2)A geometry-based feature extraction model is proposed,and the features are mainly face landmarks and Delaunay triangles.Two feature extractors,DNN and GCN,are used to extract the above two types of features,and an identity normalization preprocessing and pseudo-labeling method is proposed to reduce the negative impact of identity bias on geometric feature expression,which can greatly improve the accuracy of gaze prediction.(3)Two model fusion strategies are proposed,namely linear weighted fusion and Transformer-based fusion.According to the respective characteristics of the two fusion methods,an experiment is designed to analyze the contribution rate of each sub-model and feature,and try to find the feature with the strongest expressive ability in the task of gaze prediction.