Research On EEG And EMs Data Representation And Fusion For Mild Depression Recognition

Affective computing technologies sense the emotional state of a user and respond by performing specific features that can exert influence on emotion.Affective computing gives computer the ability to perceive,understand and express emotion.In the field of medical research,it is found that mental diseases are often accompanied by cognitive impairment and emotional abnormalities.Affective computing can provide effective methods and means for objective quantification and evaluation of these mental diseases,so it is often used for auxiliary diagnosis of various mental diseases.As a typical affective disorder,depression has a high prevalence rate and great harm.It is a major mental health problem in the current society,which seriously affects the physical and mental health of patients.In recent years,depression recognition methods based on Electroencephalogram(EEG),eye movements(EMs),brain image,voice,network behavior,which can reflect physiological and psychological changes,have been widely investigated and become a research hotspot.Modern medicine divides depression into mild,moderate and severe according to the degree of depression.Early detection and early treatment is very important for the prevention of depression.However,early symptoms of mild depression are not obvious,which often leads to low consultation rate,high rate of erroneous diagnosis,also it is difficult to detect its specificity under the current commonly used evaluation methods such as brain function imaging and voice.So current studies mainly focus on major depressive disorder(MDD)and pay less attention to mild depression.EEG is very sensitive to slight changes in brain activity,and has the advantages of high temporal resolution and non-invasive.As behavioral data,EMs data can directly reflect the psychological state.Therefore,this paper adopts the experimental paradigm of emotional face free viewing,combined with depression assessment scales and doctor consultation,to collect EEG and EMs data of college students.Single-modal EMs,EEG classification and brain function network analysis are introduced,based on which,further investigation of EEG and EMs data representation and fusion methods for mild depression recognition are highlighted.The main work and innovations are as follows:? In order to solve the problem of unstable classification effect and poor generalization performance caused by individual differences of physiological signals,a combined classification model CBEM(Content Based Ensemble Method)is proposed for the first time.CBEM is a composite model composed of multiple classifiers,during the data processing step,the data subsets are divided according to the types of experimental stimulus materials.Performance of each combination of classifier and data subset is evaluated and the final result is determined according to the voting strategy.On this basis,the static CBEM model with fixed combination and dynamic CBEM model with variable combination are proposed.The experimental results show that the CBEM model can significantly improve the recognition accuracy of mild depression and the accuracy can reach 80.75%,which is 6% higher than that of traditional classification method.? The information amount of single modal physiological data is always limited,which makes it unable to reflect the depression characteristics comprehensively.To solve this problem,this paper constructs mild depression recognition model based on bimodal data fusion strategy of Denoising Autoencoder(DAE).Using the information complementarity between the multimodal data,the information content of the input data is widened,so as to improve the recognition accuracy of mild depression.Two fusion strategies are proposed,explicit layer fusion and hidden layer fusion,experimental results show that the proposed fusion strategy can significantly improve the accuracy,and the explicit layer fusion strategy based on DAE is more effective,with the accuracy of 88.97%.? In view of the lack of investigation of data correlation in current multimodal data fusion researches,this research explores new characterization features based on the mutual information analysis of correlation between EEG and EMs data,and proposes a bimodal data fusion model MIBFM(Mutual Information Based Fusion Model)for mild depression recognition.MIBFM extracts pupil size signal from eye movement data,and selects EEG electrodes based on temporal and spatial characteristics of pupil information,so as to achieve the goal of data dimension reduction and feature space optimization.On this basis,the selected EEG data are divided into five bands: delta,theta,alpha,beta and gamma,and the pupil waveform is also divided into five bands: EM-delta,EM-theta,EM-alpha,EM-beta and EM-gamma,and the linear and nonlinear features are extracted respectively.Finally,based on the new features,different fusion strategies are selected for mild depression recognition.Experimental results show that compared with the traditional feature fusion method,MIBFM can significantly improve the fusion effect of beta,theta,alpha,delta and other bands.Among them,the effect of explicit layer fusion strategy is the best,which can achieve 88.95% of the classification accuracy of mild depression.With the aim of better mild depression recognition,this paper proposes an ensemble classification model of CBEM,which solves the problems of poor generalization performance and unstable classification effect of single classifiers.Based on the DAE,a mild depression recognition model is constructed based on the fusion of EEG and EMs data,which improves the recognition accuracy of mild depression.In addition,an EEG and EMs data fusion model MIBFM based on mutual information is also proposed.On the basis of optimizing feature space,only pupil information and a few EEG leads are used to obtain a higher recognition accuracy of mild depression.The research results give a new insight into the construction of mild depression recognition model based on EEG and eye movement bimodal data,and can also provide theoretical basis and technical support for the development of related portable and pervasive application systems.