Fatigue Annotation Methods Based On Eye Tracking Data

In order to build valid and reliable driving fatigue detection model,we need to collect multi-modal signals from drivers or vehicles,and analyze the differences of signal patterns under various drivers' conditions.To achieve that,we need to annotate corresponding fatigue level of these driving data.Among various signal sources that could characterise driving fatigue level effectively,eye movement data is one of the signals that we could obtain easily.Therefore,it is of great significance to study the method to annotate fatigue index based on eye movement data for building driving fatigue detection model.Among the research of driving fatigue based on eye movement,PERCLOS(percentage of eyelid closure)is widely accepted as a great method to monitor driving fatigue effectively,since the value is related to drivers' fatigue level significantly.We use eye movement parameters provided by eye tracking glasses to compute PERCLOS,and validate the precision by the computation method based on eye videos.We propose that PERCLOS by ETG could be served as the rough ground truth of fatigue index to evaluate fatigue detection algorithms.For this purpose,we conducted several vigilance experiments,and utilized an effective fatigue detection algorithm based on EOG data to analyze the experiment data.We used PERCLOS by ETG as the estimate of subjects' fatigue index,learning fatigue detection model from the data,and then predicted the test data with the model.Through the analysis of the experiment results,we concluded that the value of PERCLOS could reflect subjects' fatigue level to some degree,but we cannot rely on it to annotate fatigue index of driving data completely.We suppose that when PERCLOS is small enough,the driver is alert for sure,when PERCLOS is at a higher level,the driver must be fatigued,and while PERCLOS is located between these two ranges,we could not tell the condition of the driver with solely the value of PERCLOS,i.e.,the gray area.There are certain differences of several eye movement patterns of drivers under various conditions,so we can make use of these differences to analyze the eye movement data of the gray area.If the patterns of eye movement are closer to the alert ones,we annoate the corresponding data with alert label,and vice versa.We extract four types of eye movement features from the data provided by eye tracking glasses,including blink,fixation,saccade,and repetitive eye movement sequence patterns.We use the minimum redundancy maximum relevance feature selection algorithms to filter the useful eye movement features to distinguish different subjects' conditions,and use support vector machine model to learn the differences of eye movement feature patterns under different conditions.From the result of prediction on the test data,the trained model could classify different eye movement data effectively,therefore,we have sound reasons to believe that this method could annoate subjects' fatigue status of eye movement data in the gray area reasonably.