Design And Implementation Of Multi-mode Eye Movement Data Visualization Software

Objective: Eye movement research is an important means to study human cognitive psychology.Visualization of eye movement data is an indispensable method in the study of eye movement.Most of the existing commercial eye movement data visualization software is expensive,cannot be visualized in real time and requires manual selection of stimulus materials to regenerate visual results,while most of the free software can only generate visual results of image eye movement.Therefore,this paper aims to design a multi-modal eye movement data visualization software.Considering that the current methods of acquiring external information are mainly images,text and video,this paper designs and realizes the real-time eye movement thermal map to show the real-time attention position of users when they watch these three modal information.After the interaction with the software,According to the corresponding eye movement data,the visual results of image,text and video are automatically generated.Methods: The software was divided into interface interaction module,eye movement data acquisition module,eye movement data visualization module,recording and playback module.The interface interaction module mainly includes the use of Qt to present multi-modal information,real-time thermal map display,etc.The function of the eye movement data acquisition module is to obtain the eye movement data collected by the Tobii 4C eye tracker through the API provided by the Tobii official website.Eye movement data visualization module is used to generate eye movement heatmap and scanpath.The generation method of eye movement heatmap is to calculate the density value of any pixel point of the observation target affected by eye movement data point by kernel density estimation method,and then correspond the density value with the color mapping table to obtain the heatmap.The way to generate the eye movement scanpath is to filter the eye movement data according to the definition of the eye movement mode,then determines the effective observation area according to the filtered eye movement data,and finally connects the adjacent effective observation areas with lines.The recording and playback module realizes the function of recording screen by FFmpeg tool.Results: The software designed in this paper can realize the corresponding function.The designed and realized real-time heatmap can reach the refresh rate of 30 frames per second,and there is no obvious stutter on the interface,which meets the requirement of real-time performance.After the interaction between the user and the software,the heatmap and scanpath are generated according to the corresponding eye movement data.The warmer color in the static heatmap also corresponds to the dense area of eye movement data points in the scatter map of eye movement data,which can reflect the spatial distribution of visual attention,and the eye movement scanning path can reflect the sequence of human eyes.Conclusion: The multi-modal eye movement data visualization software designed in this paper can generate eye movement visualization results of image,text and video based on the collected eye movement data.Compared with most visual analysis software,it has the function of real-time heatmap display,automatic generation of visual results and visual analysis of video,which has certain practical application value in the field of eye movement research.