EEG And Eye Movement Data Fusion Study For Mild Depression Based On Individual Difference Elimination

Mild depression is more common in daily life and has great hidden dangers,but it has not attracted enough attention,and research related to mild depression is still in its infancy.Early intervention can avoid the severity of depression.However,the current mainstream depression diagnosis is mainly based on the combination of questionnaire scale and doctor’s consultation,which is subjective and time-consuming.Therefore,an objective and quantified identification method for mild depression is urgently needed to assist the diagnosis.In recent years,the development of physiological signal monitoring and computer technology has opened up new ideas for depression recognition.Among them,Electroencephalogram(EEG)has been widely studied due to its advantages such as high temporal resolution,non-camouflaging,non-invasive,etc.However,just like fingerprints,EEG has individual differences that cannot be ignored,which affects the generalization performance of the model,so it is worth exploring how to eliminate the individual differences based on the EEG depression recognition model and make the model more generalized.Eye movement signal has attracted much attention because it’s easy to acquire,low cost and can reflect people’s cognitive ability and attentional bias.At present,many studies only identify depression based on single-modality physiological data,but the coverage of single-modality signal is limited and cannot reflect the comprehensive characteristics of depression.Therefore,it is becoming a trend to fuse physiological signals from multiple modalities to build a more comprehensive and accurate depression recognition system.In summary,based on the free browsing paradigm of emotional pictures,this paper simultaneously collects EEG and eye movement data of 20 patients with mild depression and 20 healthy controls,and analyzes and processes them with computer technologies such as transfer learning and data mining to explore the specific indicators of mild depression.This paper aims to eliminate the influence of individual differences in EEG,and use multimodal fusion to seek a diagnostic method of mild depression with strong generalization,high robustness and objective quantification.The main research content and innovation points of this paper are as follows:(1)Aiming at the problem that individual differences affect the generalization of the EEG model in depression recognition,the spatial distribution of EEG characteristics of two groups of people was explored and studied,and multiple methods of transfer learning were used to eliminate the influence of individual differences in EEG,which improved the classification performance of the model.In this study,using the all-band features of EEG,based on ten-fold cross-validation,transfer component analysis(TCA)was first used to minimize the edge distribution of the training and test sets,and secondly,joint distribution adaptation(JDA)was used to minimize the joint distribution of the training and test sets.Finally,balance factor μ was added to explore the effect of spatial distribution of EEG features on depression recognition by eliminating different proportion of marginal distribution and conditional distribution.The results show that the classification accuracy of linear nonlinear features can reach 77.50% in logistic regression after eliminating individual differences,which is 12.56% higher than the original features.The highest network feature can reach 89.10% on SVM,which is 7.75%higher than the original feature.Similar results and improvements were obtained for the two types of EEG features in the experiment,which verified that eliminating individual differences in EEG based on transfer learning is conducive to improving the accuracy of mild depression identification and model generalization performance.(2)Aiming at the problem that single-modality is difficult to fully represent the characteristics of mild depression,this paper proposes a hybrid fusion model based on DBN and secondary classifier(HFMDBSC),which effectively integrates various types of EEG and eye movement features,eliminates individual differences in EEG,and improves the accuracy of mild depression recognition.The model first uses deep belief network(DBN)in the feature level to fuse two types of EEG features: linear nonlinear features and network features.DBN transfers features to hidden layer space,which eliminates individual differences existing in EEG and fuses both types of features.After that,DBN features and eye movement features are fused at the decision level using a secondary classifier at the decision level.Combine the decision information of the two modes as the final result.The results show that the DBN features after feature layer fusion are significantly improved compared with linear,nonlinear features and network features.After combining the decision-level results of the two modalities,the accuracy of mild depression recognition has been further improved,reaching a maximum of89.54%,which can effectively identify mild depression.In conclusion,this paper eliminates individual differences of EEG based on transfer learning method,and solves the problem that individual differences of EEG affect model generalization performance.A multimodal mild depression recognition model HFMBDSC is proposed,which gives a new idea for multimodal depression recognition.On the basis of eliminating the individual differences in EEG signals,a variety of decision-making information was combined to improve the correct rate of identification of mild depression.The results can provide theoretical basis and technical support for the elimination of EEG individual differences and the construction of multimodal depression recognition models.