Gaze Estimation For Multiple Data Sources

Numerous social psychological studies have shown that the gaze usually plays an important role in attention analysis and interpersonal communication.After more than 30 years of research,researchers have proposed various gaze estimation solutions and have produced high-precision commercial systems.However,most commercial systems require intrusive dedicated hardware and usage scenarios are limited.Therefore,the low cost,non-invasive and suitable for any head posture gaze estimation algorithm becomes the main research direction of current gaze estimation.In this thesis,the depth image is added to the color image to improve the accuracy of the head pose estimation,and the application range of the infrared image addition algorithm is added.Therefore,a method for gaze estimation for multiple data sources is proposed.The gaze estimation for multiple data sources is based on the collected depth image,color image and infrared image for head pose estimation and pupil positioning,so as to perform gaze estimation.The basis of gaze estimation is accurate head pose estimation and pupil location,so head pose estimation and pupil positioning are the focus of this thesis.For the head pose estimation problem,this thesis combines the geometry-based method with the learning-based method for head pose estimation.On the basis of face detection and face alignment,the geometric features of the color image or the infrared image and local region depth features of the depth image are extracted,and then the normal and curvature features of the depth block are combined to form a feature vector group;then the random forest method was used for training.Finally,all decision trees are voted,and the obtained head pose Gaussian distribution estimation is threshold filtered to further improve the accuracy of the model prediction.For the pupil positioning problem,different methods are used to locate the pupil in different data source images.In the color image,the pupil template is used to locate the pupil.In the infrared images,the images can be divided into bright pupil images and dark pupil images according to the position of the camera.Different image processing methods based on pixel features are used in the two kinds of images for pupil positioning.For the gaze estimation problem,this thesis used the coordinate transformation method to analyze the global gaze offset caused by the head pose and the local gaze deviation caused by the pupil position,and transforms the global feature and the local feature into the coordinate system to estimate the world coordinate system.In summary,based on the current algorithm and related processes of the gaze estimation,this thesis implements the proposed gaze estimation algorithm for multiple data sources through the provided experimental platform.The method has been verified on the public data set and local data sets.Under the premise of ensuring real-time performance,the accuracy of the gaze estimation can be improved.