Reserch On Train Driver’s Fatigue Evaluation And On-line Detection

With the development of railway transportation productivity, higher requirements for railway traffic safety have been requested. Since train driver’s fatigue is one of the main reasons to cause railway traffic accidents, the railway management department has the urgent needs to prevent train driver’s fatigue by applying advanced technologies and equipments, to further reduce the human accidents caused by train driver’s fatigue. Under this background, this paper takes the train driver as the research object and focuses on key issues in studies of the fatigue formation mechanism, impact factors & risk assessment, features extraction & analysis, and multi-information fusion detection of the train driver’s fatigue. Based on these studies, an accurate and robust real-time system for train driver’s fatigue on-line detecting and early warning has been designed and implemented. The specific research contents are as follows:1. Aiming at the particularity of train driving, the driver’s fatigue formation process and mechanism was systematically studied by investigating their operation behaviors, crew system and working conditions. Thus, a variety of fatigue impact factors including train driver’s workload, shift work, professional quality and working environment were summarized to provide the basis of the fatigue risk evaluation. Meanwhile, the behavioral physiological and psychological performance characteristics of the train driver under fatigue state were discussed for investigating the applicability and feasibility of these performance characteristics for driving fatigue detection.2. The train driver’s workload calculating method was proposed, based on hierarchical decomposition of train driving tasks and estimation of the task demands of human body. Through the study of the task demands of human body under concrete train driving works, this method can reflect the driver’s workload objectively and accurately. Subsequently, an improved Fuzzy Analytic Hierarchy Process (FAHP) method was used to combine the multiple fatigue impact factors (from workload, shift work, professional quality and working environment aspects) to establish a train driver fatigue risk evaluation model. Eventually, the effectiveness and accuracy of this evaluation model were verified by the fatigue risk evaluation results of the train drivers during different periods in different trips. The proposed evaluation model can not only be used to avoid train driver’s fatigue by shift management, but also provides the prior knowledge for the train driver fatigue detection based on information fusion.3. In this part, a train driver fatigue on-line detection scheme was established based on audio and video multi-sensor information fusion and the fatigue-related features were extracted and analyzed. Firstly, according to the influence of human fatigue on vocal system, the traditional speech features (e.g. pitch frequency, formants and MFCC) and non-linear features (e.g. the largest Lyapunov exponent, fractal dimension and ApEn) of the speech chaotic non-linear dynamic model were extracted to analyze the variation of these speech features with human fatigue. In addition, the correlation between the speech features and the driver fatigue was verified by experiments. Secondly, in order to extract the eye and facial features from train driving video data quickly accurately and robustly, a series of image processing & recognition algorithms (such as:a skin-color segmentation algorithm based on skin-color adaptive update model; a Haar-Adaboost based face recognition algorithm; a eye location and status recognition method based on Zernike moments, et al.), which can be used to narrow the feature extraction ROI effectively, were proposed to solve the problems caused by illumination changes and head rotation in the real train driving environment. Last but not least, the statistical regularity of eye features (e.g. PERCLOS, AECS, MECD, AOL and BF) and facial features (e.g. YawnFreq and NodFreq) of the train drivers in different fatigue states were analyzed.4. A multi-information fusion method combined feature-level fusion with decision-level fusion was proposed for detecting train driver’s fatigue by using the fatigue risk evaluation results and the driver’s multifaceted features. In regards to the feature-level fusion, the multi feature fatigue classifier was designed based on the Fuzzy Support Vector Machine (FSVM) algorithm. And a new fuzzy membership function was used in this algorithm to improve the feature-level fusion classifiers. In regards to the decision-level fusion, a Dynamic Bayeian Network (DBN) probabilistic inference method was proposed to fuse the decision results of four feature-level fusion classifiers (speech features, eye features, YawnFreq and NodFreq) with the fatigue prior knowledges (reflected by the fatigue risk evaluation results) for obtaining the final fatigue detection results by DBN probability inference. Experimental results show that this multi-information fusion detection method could adaptively adjust the fatigue decision criteria according to the prior knowledge of fatigue risk, and effectively reduce the influence of each feature-level classifier’s false detections on the final detection results. Therefore, this method shows outstanding detection performances.