Visual Saliency Detection Of Night Traffic Scene Based On Deep Learning

The traffic driving environment is a complex and changeable dynamic scene with intricate information.Affected by the visual selective attention mechanism,experienced drivers can quickly search the key information of driving task from tremendous amount of traffic scene information,and then make predictions to ensure driving safety.Recently,people pay more and more attention to the visual saliency detection technology of traffic scenes.By analyzing and simulating the driver’s attention distribution,the saliency regions and targets in the driving scene can be predicted.At present,most researches focus on the prediction of daytime traffic scenes.However,due to insufficient light and interference from light sources,night driving scenes are more complicated and dangerous than daytime.Modeling of driver’s attention distribution in night scene can improve the safety performance of human-centered driving assistance system.The main content of this thesis concludes the following three parts:Firstly,we constructed a video dataset for saliency detection in night traffic scenes.We analyzed the limitations of current traffic saliency datasets,and then designed an eye movement experiment for night traffic scenes.After that,we recruited 30 experienced drivers to record the eye tracking data while they were watching video by simulating daily driving.It is worth mentioning that the dataset contains bottom-up exogenous stimuli and top-down driver endogenous attention information.Secondely,on the dataset,we proposed a dynamic visual saliency model NTDSD based on convlstm.Conv LSTM is employed to extract temporal information of traffic video,pyramid technology is adapted to enhance spatial features,and attention mechanism is used to fuse spatial and temporal features more effectively.Compared with other models,this model is more accurate and concentrated in predicting the main targets that drivers are concerned about.In addition,the model can effectively capture the motion characteristics of small objects such as pedestrians and bicycles,and has more outstanding performance when detecting pedestrians and bicycles in weak light.Finally,we established a visual saliency prediction method NTDSD-VA for night traffic scene based on spatiotemporal audio-visual information.We builded a visual module and an auditory module to extract video and audio features respectively,and tried to integrate the two features into the prediction network.The results showed that the model can accurately predict the main and secondary goals related to driving tasks.We further designed an ablation study to explore the contribution of auditory module,and analyze the reasons why the auditory module did not achieve the expected effect.The new dataset we constructed can be used for follow-up study about attention prediction via deep learning model in such traffic driving scenes.At the same time,the visual dynamic detection model and audio-visual spatiotemporal detection model proposed in this thesis make varied attempts to detect the visual saliency of night driving scene,which may provide alternative research direction for the subsequent saliency detection.