Detecting Driving Fatigue With Multimodal Deep Learning

With the rapid development of portable physiological signal acquisition equipments,signal processing algorithm and machine learning algorithm,electroencephalogram(EEG)and electrooculogram(EOG)signals have been successfully used for driving fatigue detection separately.This study proposes a new method of combining partial EEG and forehead EOG for detecting driving fatigue.We design a simulate driving experiment to record EEG,EOG and eye movements at the same time.We also do experiments on the effect when EOG is combined with different parts of EEG,and point out the key fusion brain area.Our experimental results demonstrate that we can extract the most effective shared features from the combination of the six-channel EEG signals from temporal EEG and the forehead EOG.Furthermore,this study proposes a multimodal method based on the deep autoencoder model to extract the fusion features.We compare the proposed multimodal method and traditional single modality methods on twenty-two different subjects on driving fatigue detection.It is shown that the multimodal fusion method can get a relatively better performance for driving fatigue detection compared with single modality.The average COR and RMSE of the multimodal method achieves 0.85 and 0.09,respectively.