Research On Visual Attention Mechanism Of Dynamic 3D Scenes Based On Deep Learning

Three-dimensional display technology can provide the viewer with a three-dimensional picture containing real scene depth information.In recent years,with the development of the new generation of information technology,the application of high-performance three-dimensional display devices in various research fields has become increasingly urgent.However,the existing 3D display technology is limited by hardware conditions such as insufficient computational resources and low resolution of the LCD display.As a result,the display quality has not been able to reach a high level.Starting from the visual characteristics of the human eye,this essay investigates the visual attention mechanism for dynamic 3D scenes,predicts the region of interest to the human eye,so as to effectively allocate computational resources and use the limited computational resources to display the region of interest in a higher quality and improve the 3D display effect.Since in most practical application scenarios,such as medical imaging,military exercises,scientific research and 3D games,the viewer is driven by a specific task during the viewing process,and the gaze behaviour is controlled by subjective consciousness,and the scene is generally dynamic,this essay establishes a task-driven 3D dynamic scene eye-movement dataset,and investigates the features of the dataset to verify the validity of the dataset Secondly,a neural network is used to construct a human eye visual attention model to predict the human eye gaze area in dynamic 3D scenes under task-driven conditions.The main contents and innovations of this essay include:1.Establishing task-driven eye-movement dataset for dynamic 3D scenes:Most of the existing eye-movement datasets are based on 2D images or 2D videos,in order to address the lack of 3D video eye-tracking datasets,this topic designs and shoots 3D video stimulus materials,asks subjects to perform a specific task while watching the video,records subjects’ eye-gaze data under the task-driven condition with eye-tracking devices,and then processes the data to obtain the density map of human eye gaze,which was used as the ground truth for optimising the network in subsequent modelling to predict the human eye gaze points;secondly,the data set was quantitatively analysed in two dimensions to verify its validity in the subsequent modelling process,including the temporal correlation of attention on consecutive frames and the correlation between human eye attention and moving objects.2.The prediction model of human eye attention area for dynamic 3D scenes is proposed:in realistic scenes,the depth information of objects is very important for human eye attention,which can provide the human eye with the spatial orientation information of the target and thus influence the distribution of human eye attention.Therefore,when building a prediction model for the human eye’s gaze area in 3D scenes,an effective integration of depth information with colour information will help to improve the prediction effect.Based on the classical method of modelling the human eye attention mechanism in 2D scenes,this topic adds depth information to the network and constructs a dual-stream multi-scale feature extraction module and a depth-induced multi-scale weighting module to extract spatial information and simulate the bottom-up attention mechanism driven by the video information itself;secondly,a temporal feature extraction module consisting of a two-layer convolutional long-short term memory network is constructed to learn the temporal transfer relationship between target objects,consider the temporal correlation of gaze points between video frames at the pixel level,simulate the task-driven top-down attention mechanism,and finally output the prediction map of human eye gaze areas in dynamic 3D scenes.The final prediction results of this subject model and the real human eye gaze density map reach a high degree of similarity,and the performance of this model in comparison experiments is also excellent.The results can guide the effective allocation of computational resources for 3D display,enhance the 3D display effect,and have certain guiding significance for the development of high-performance 3D display technology.