Saliency Area Detection Of Rainy Traffic Video Based On Deep Learning

The traffic environment is an extremely complex dynamic scene with multiple information sources,including targets that are highly related to the current driving task and redundant targets that interfere with us.Experienced drivers can quickly complete the screening and filtering of information under the action of the visual selective attention mechanism,and focus their attention on prominent areas such as vehicles,pedestrians,motorcycles,traffic lights,and traffic signs to ensure driving safety.Especially when there is rainy weather,the driving task becomes more difficult due to factors such as low visibility,slippery road surface,rain falling and umbrella blocking pedestrians.In addition,the traffic environment changes rapidly,and once some important goals related to driving safety are ignored,serious consequences may result.Modeling driver eye movement and attention allocation during driving in rainy conditions can help guiding unmanned intelligent vehicles and improve the traffic safety during similar conditions.However,there are few studies at present.Based on this,we collected an eye tracking dataset from 30 experienced drivers while they were viewing 16 traffic driving videos under a hypothetical driving condition,and proposed corresponding saliency models to predict the driver’s gaze when driving on a rainy day.The contents are divided into three tasks shown as follows.Firstly,based on the current situation that the video dataset is insufficient,we collected 16 rainy traffic driving videos and designed eye movement experiments.It recorded 30 drivers while they were viewing the video in the simulated driving task under the voice navigation.Then,a large-scale dynamic eye tracking dataset(RTD)has been established,which provides a data foundation for the research in the field of traffic scenes.Secondly,according to the dual-branch theory of information transmission in the visual cortex,this thesis effectively combined the visual attention mechanism with deep learning technology to establish a dual-branch rainy traffic video saliency detection model.The experimental results show that the model proposed in this chapter could predict the driver’s main focus area and other secondary goals that are highly related to the driving task well.Finally,in view of the potential advantages of 3D convolutional neural network processing videos and the shortcomings of the Tased-net model,we improved the model by adding optical flow information,pre-training on the optical flow image to fully learn the motion information between video frames.After that,the model was fine-tuned on the RTD dataset through the transfer learning,so that the model can extract the image spatial features more refined.The analysis and comparison shows that the improved model is more sensitive to the moving target while the accuracy is not reduced,and is closer to the saliency map of the human.This thesis is based on the top-down visual attention mechanism,and the two saliency detection models proposed could predict the saliency area in the rainy driving dynamic scene well,which provides technical support for the development of assisted driving systems and intelligent driving vehicles.In addition,the dynamic eye tracking dataset established in this thesis is of great significance to promote the application of deep learning technology in the field of video saliency detection.