Detection And Analysis On Driver Fatigue Based On Multimodal Physiologycal Information Aimed For ADAS

Driver fatigue is one of the top causes of traffic accidents,since the fatigued drivers are unable to adequately perceive,react and respond to situations on the road.The study on driver fatigue has become interest of researchers worldwide.Aimed for building a human-car interacted advanced driver assistance system(ADAS),the detection and prevention for driver fatigue have been investigated worldwide.The front discrimination sub-system of ADAS may be divided into two components,perception component and discrimination component.In perception component,how to receive the reliable multimodal information?And in discrimination component,how to accurately discriminate the driver state?These two questions are the key points for the development of ADAS system.Human behavior is controlled by the nerve system.Electromyography(EMG),Electroencephalogram(EEG),and Electrocardiography(ECG)may reflect the human body.And comprehensive analysis on the drivers'multimodal physiological information can improve the accuracy for driver fatigue detection,and can give a theoretical foundation for the development of ADAS.Therefore,under the conditions of actual and simulated driving,susceptible positions of cervical and lumbar muscles were determined by biomechanical analysis,and the variation of physiological features of different muscle positions was investigated.Then the physiological signals and biomechanics were combined to be multimodal physiological information,and its discrimination on driver fatigue was investigated.And the optimized characteristic features were obtained,which can improve the reliability of perception component of ADAS.Then,a series of discriminated models on driver fatigue were built,which can improve the accuracy of discrimination component of ADAS.The main findings and innovations in this thesis could be summarized as follows:(1)Based on the biomechanics,a fatigue index Q combined by a linear index(Integrated electromyography,IEMG)and a nonlinear index(Sample Entropy,SaEn)was proposed.With the increased time of neck-flexion,IEMG of surface EMG increases,SaEn and index Q decrease.In the later stage,the variation rate becomes slower,indicating deeper fatigue of muscle.The results of paired-t and variation coefficient show that index Q has the best discrimination due to a)the obvious difference,b)the stable data,c)being consistent with biomechanical analysis,which can describe the fatigue degree of different muscle positions.These results are consistent with cervical biomechanical mechanism,the muscle at 6th cervical vertebra is easier to be fatigued than the 7th.Therefore,index Q can better describe the fundamental information of data and the variation of EMG,also can compare the fatigue degree of different muscle positions.And it is an effective characteristic feature to evaluate fatigue degree of cervical muscles.(2)Based on the biomechanics,the susceptible positions of muscles were scientifically and reasonably determined,which is benefit for the determination of international standard positions of EMG extraction.A biomechanical model is built based on driving posture and cervical and lumbar musculoskeletal structure,which can determine the susceptible positions of cervical upper trapezius and lumbar vertical rachial muscles.The upper trapezius of 6th cervical vertebra and the vertical rachial of 4th lumbar vertebra are sensitive and easier to be fatigued.The normal and fatigued state can be effectively discriminated if the complexity of cervical EMG and the complexity and approximate entropy(ApEn)of lumbar EMG are combined.These cervical and lumbar features can be effectively combined by principle components analysis(PCA),retaining useful information and eliminating redundant one.Based on multi-regression theory,an effective mathematical model(CL-EMG model)on the discrimination for driver fatigue was built.The accuracy and robustness of discriminated model are higher,because the biomechanical characteristics and physiological features were combined to be multimodal variables.Based on state validation and cross validation,the accuracy of this model is up to 91%.(3)Feature analysis on multimodal physiological information.In case of actual driving on highway,different signal combinations were comprehensively analyzed by Fuzzy-C clustering.Complexity and SaEn of cervical muscle,lumbar muscle and EEG decrease as the extension of driving time.After about 90 minutes,the decreasing rate become slowly,indicating deeper driver fatigue.In the process of feature extraction,the optimized multimodal signal combination was obtained by retaining useful information and eliminating redundant one,which can effectively discriminate driver fatigue.Then,the multimodal information and reasonable dimensionality were effectively achieved,which can give a theoretical guide for accurate discrimination of driver fatigue.Based on multi-regression theory,cervical-lumbar EMG and EEG were combined,and a mathematical model(ML-EMG/EEG model)on the discrimination for driver fatigue was built.By the state validation and cross validation,the accuracy of the model is up to 95%.This can improve the reliability of perception component and accuracy of discrimination component in front sub-system of ADAS.(4)Based on the above contact sensors,a portable real-time and non-contact sensor was proposed and applied in driving experiments.The sensors were located under the driver seat,and under the non-disturbance condition,the mixed multiple physiological signals of the biceps femoris of driver were recorded,including EMG,ECG,and artifacts.In this original mixed signal,EMG and ECG can be effectively separated by fastICA,and denoised by empirical mode decomposition(EMD).Then,four features,including complexity and SaEn of EMG and ECG,are extracted and analyzed,and all of them decrease with the increase of driving time.Three principal components are obtained by PCA and are used as independent variables.Finally,two mathematical models on driver fatigue were built,a)PCA-ML model based on multi-regression theory,and b)PCA-Bayes model based on Bayes theory.By the state validation and cross validation,the accuracy of these two models may be up to 89%and 92%.Above all,four discriminated models on driver fatigue,CL-EMG?ML-EMG/EEG?PCA-ML?PCA-Bayes,were built in this thesis,and their accuracy were 91%?95%?89%?92%respectively.Among them,multimodal physiological information,including biomechanics and physiological signals,was considered in ML-EMG/EEG model.Also,PCA and SaEn were used,and the multimodal information and reasonable dimensionality were effectively achieved,so its accuracy is the highest.In PCA-Bayes model,a portable non-contact sensor was used to record the physiological signals of driver.There is no disturbance on driver.So it is easy to be used in actual driving,and the accuracy is higher too.Therefore,the experimental results have great significance for the development and application of ADAS.In order to avoid traffic accident,when the driver's information is reliably obtained in perception component of ADAS,and driver fatigue state is accurately discriminated in discrimination component,the vehicle can be switched to assistant control and operate mode.So the research results of this thesis have great theoretical and actual significance on driver fatigue and ADAS.